STOCHASTIC    PROCESSES USED IN TIME SERIES MODELING Time series   data set consist of observations on a variable or several varia...


Time series  data set consist of observations on a variable or several variable overtime. Example of time series data include stock prices, money supply, consumer price index, gross domestic product, annual homicide rates ,and automobile sales figures.

A key feature of time series data that makes them more difficult to analyze than cross sectional data is that economic observations can rarely, if ever, be assumed to be independent across time. Most economic and other time series are related, often strongly related, to their recent histories. For example, knowing something about the gross domestic product from last quarter tells us quite a bit about the likely range of the GDP during this quarter, because GDP tend to remain fairly stable from one quarter to the next.

Another feature of time series data that can require special attention is the data frequency at which the data are collected. In economics, the most common frequencies are daily, weekly, monthly, quarterly, and annually. Many macroeconomic series are recorded   monthly, including inflation and employment rates. Other macro series are recorded less frequently, such as every three month (every quarter). Gross domestic product is an important example of a quarterly series. Other time series, such  as infant mortality rate, are available only on annual basis.

This section presents different types of stochastic process that have been found useful in time series modeling .They include:

 Purely random process

A purely random process is a general process underlying the distribution of all possible outcome that can occur overtime. The distribution at every point in time is represented by the graph. It is also called a stochastic process. A random purely random process is a discrete process (Xt ), consisting of sequence of mutually independent and identically distributed random variable. Its unconditional mean and variance are constant:  E(Zt )= E(Zt+k)= µ       var(Zt)= var( Zt+k)= ơz2. Its auto covariance function (acvf), ƴ(k) + cov(Zt,Zt+k)=0. The covariance depends only on the time distance between two period and not time itself.

Note that a purely  random process is also referred to as WHITE NOISE. This stochastic process is clearly  stationary.

Autoregressive (AR) process

It is a regression model where the independent variables are lags of the dependent variable (an autoregression is a regression of a variable on lag of itself ). Assume that (et) is a purely random process with mean =0 and variance = ơ2 .Now, consider the   stochastic process (Zt) given by

Zt=£+ Ɣ1Zt-1 + Ɣ2 Zt-2+…+ ƔpZt-p + et        [1]

This stochastic process Z, is known as autoregressive process of order p and is written as AR(p). Basically, an autoregressive process depends on a weighted sum of its past values and a random disturbance term. P refers to the order (maximum lag) of the process.

Let see a simple example of autoregressive model with the explanatory variable which is one period lag of dependent variable. This is called the AR(1) model.

Zt=£+ Ɣ1Zt-1+et , for t=2,…,         [2]

Ɣ =1 implies the type of trend behavior is non stationary.  The other values of  Ɣ imply stationary behavior. This allows us to provide a formal definition of the concept of stationary and non stationary ,at least for the AR(1) model, we can say that Z is stationary if / Ɣ/< 1,and non stationary if Ɣ=1.The other possibility, / Ɣ/ >1, is formally, “non-stationary”. At this stage it is useful to think of a unit root as implying Ɣ=1 in the AR(1) model.

Random walk process

Consider the case where Ɣ=1 and £=0 from equation 2. In this  case, the AR(1) model  can be written as :

  Zt= Zt-1+et           [3]  

This is referred to as the random walk model. Since Ɣ=1, Zt becomes a random walk with drift. It has unit root and is non stationary. If however, Ɣ< 1, then the series Zt   is stationary. In the area of finance, the model of random walk process is often used to describe the behavior of stock market process. The economic meaning of this is that though stock market prices are non stationary rising over time, the change in share price will be purely random process.

Moving average (MA) process

Assuming that et is a purely random process with its mean equal to zero and variance  equal to ơz2.Now,if

Zt=µ + et+ ƴ1et-1+et-2+…+ƴqet-q    [4]

Then, the stochastic process (xt) is called a moving average process of order q and is written as MA(q). In other words, a moving average process may be obtain as a weighted sum of current and lagged random disturbance. It is a function of the difference disturbances; current disturbances, lag disturbance, up a maximum lag of q.

The MA(1)(first order moving average) process is given by the following equation:

Zt=µ + et+ ƴ1et-1    [5]

It means that current value of dependent variable Zt is a weighted average of current and previous value of error term et and et-1 .  The value of error term in time t( et) is define  as the white noise process et.

The sample moving model says that observation that deviate two or more periods are not

Zt= µ + et-1    [6]

This can be interpreted as the moving average representation of the autoregressive process. Autoregressive process is written as an infinite order moving average process If /Ø/<1.

It has been notice that moving average processes arise in econometrics mainly through trend elimination techniques. The pre- eminent technique use to eliminate trend is successive differencing of a non stationary time series. However, it has been found that while successive differencing  in fact succeeds in eliminating the trend, it nevertheless introduce  a  moving average process that can exhibit a cycle. In effect, the detrended series can exhibit a cycle even when there was no cycle in the original series. This unintended introduction of spurious cycle is called the slutsky effect.

Autoregressive Moving Average (ARMA) Process

These classes of models are obtained as contribution of AR and MA models and are known as ARMA models. Consider a stochastic process (zt) specified as

AR(p): Zt=£+ Ɣ1Zt-1 + Ɣ2 Zt-2+…+ ƔpZt-p + et

MA(q): Zt=µ + et+ ƴ1et-1+et-2+…+ƴqet-q

ARMA(p,q) : =£+ Ɣ1Zt-1 + Ɣ2 Zt-2+…+ ƔpZt-p + et+ ƴ1et-1+et-2+…+ƴqet-q  [7]

This class of models is particular interest as the models result in a parsimonious representation of higher AR(p) or MA(q) processes. The general ARMA model  can be also written as:

Ɣ(L)Zt= ƴ(L)et   (8)

             Estimation and Testing AR, MA and ARMA models

After fitting an AR, MA and ARMA  models to a set of time series data, it is necessary to check whether the model provides an adequate description of the empirical data. Two criteria are frequently used to test the adequacy of the goodness of fit and the appropriateness of the number of parameter estimated. They are

(i)                  Akaike information  criterion(AIC)

(ii)                Schwarz information criterion(SIC)

Given that p is the number of parameter s estimated, then we have these formulae

ü  AIC(p)= T  In(SSR) + 2(p+q+1)

ü  SIC(p)=T    In(SSR)+(p+q+1) In(T)

The objective is to choose the model with lowest AIC or SIC

The object is chose the lowest AIC or SIC. It is obvious that these criteria correct the adjusted coefficient of determination by degree of freedom.

Autoregressive Integrated Moving Average (ARIMA) Process

It is generally known that most economic time series  are non stationary. A procedure commonly used to achieve stationary is successive differencing. Let the difference operator ∆= 1-L, then ∆Z1=Zt-Zt-1,2Z1=(Zt-Zt-1)-(Zt-1-Zt-2)  and so on.

Now, if ∆dZt is stationary series that is representable as an ARMA(p,q) process,  then, it follows  that Zt is an ARIMA(p,q) model . ARIMA means an autoregressive integrated moving average process. The term “integrated” is used to denote the fact that it is “difference” data that are fitted to the stationary ARMA model. In effect, the difference data actually fitted to the ARMA model have to be “integrated” or summed before obtaining the original non stationary data time series.

An examination of the ARMA(p,d,q) model shows that it is very general.  It can be shown that a combination of a small p and q can generate a wide range of time series models. In any case, the pure random process, the moving average process and the autoregressive moving average process are all special cases of the ARIMA process. For example, if p=d=q=0, the ARIMA model collapses to a purely random process or white noise. If d=0, the ARIMA model collapsed to an ARMA process. IF d=q=0 , the ARIMA becomes an AR process , while if d=p=0, the ARIMA collapses to an MA model .

The Box and Jenkins approach in 1970 serve to popularize the ARIMA process. The ARIMA was used as a versatile and flexible instrument for forecasting. It was welcome because it laid emphasis on dynamics, nevertheless abandoned the use of explanatory economic variables. The weakness of this model is that it rely on past behavior of dependent variable alone to make forecast. If everything depends entirely on the past behavior of key macroeconomic aggregates, what is the role of policy and of policy makers?

Vector Autoregressive  (VAR) models.

VAR models are versatile and widely used tools in empirical macroeconomics and finance. VAR are reduce form time series models of the economy. The argument of VAR proponents is base on the need to study the important dynamic characteristics of the economy without prior imposition of structural restriction from any particular theory.

Although, VARs are not the same as simultaneous equation models (SEMs), they are classified as either  endogenous or jointly determined as we find in SEM. The forecast of the future of a variable therefore is based entirely on the previous value of the relevant explanatory variable.

The VAR treat each endogenous variable in the system as a function of lagged value of all endogenous  variable. To illustrate, assume that we wish to obtain forecast of two variable X (lot price) and Y (house price).

 Xt= ɑ0+ ɑ1 Xt-1 + ɑ2 X t-2 +…+ ɑkXt-k1 Yt-1 + ɓ2Yt-2 +…+ ɓkYt-k+ Ut     [9]

Yt=  Ɣ0+ Ɣ1 Xt-1+ Ɣ2Xt-2 +…+ ƔkXt-1 + Ω1Yt-1 + Ω2Yt-2+…+ ΩkYt-k+ et     [10]

It is assumed that all the variables in the VAR are stationary. Xt in equation above in current period is specified as a function of lagged values of both itself and Yt plus the error term[Ut]. The error term is assumed to be white noise. In the same way, Yt is a function of lagged values of Xt and lagged values of  itself plus the white noise error. The equations above is not simultaneous equation because Xt is not present in its equation and Yt  is not present  in its equation.

The VAR equation above is two dimensional VAR and it contain lags up to order k. If three variables where used, say, X, Y and Z in the specification, we would have three dimensional  VAR

The reason to use VAR are:

·         It is easy to use

·         It provides framework for testing for  granger causality between each set of variable.

Economic theories or common sense help to interpret the result obtained through VAR model. For instance, X caused Y or X influence Y. In both cases, it is not plausible for us to say that Y influenced or caused X.

One of the drawbacks in VARs is they are not theoretical which are not base on economic theory. There is theory in selecting the variable.

The forecasting performance of a VAR model is better than sophisticated macroeconomics models. This  is a strong motivation for using VARs. 


Ergun, U. & Goksu, A. (2013). Applied Econometrics With Eviews  Application. International  Burch University. P. 205-225.

Iganiga, B.O.(2009). Introductory   Econometrics: Made Easy. Antitop Books. P.151-158.

Wooldridge, J.M. (2013). Introductory Econometrics: A Modern Approach. Cengage Learning EMEA.


Popular Posts


Loaded All Posts Not found any posts VIEW ALL Readmore Reply Cancel reply Delete By Home PAGES POSTS View All RECOMMENDED FOR YOU LABEL ARCHIVE SEARCH ALL POSTS Not found any post match with your request Back Home Sunday Monday Tuesday Wednesday Thursday Friday Saturday Sun Mon Tue Wed Thu Fri Sat January February March April May June July August September October November December Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec just now 1 minute ago $$1$$ minutes ago 1 hour ago $$1$$ hours ago Yesterday $$1$$ days ago $$1$$ weeks ago more than 5 weeks ago Followers Follow THIS CONTENT IS PREMIUM Please share to unlock Copy All Code Select All Code All codes were copied to your clipboard Can not copy the codes / texts, please press [CTRL]+[C] (or CMD+C with Mac) to copy