1.

Random signals and stochastic processes

1.1

Statistical Estimation

rand(m, n) is a Matlab

function that generates a matrix of size m x n, containing random numbers

uniformly distributed between 0,1.

Figure 1:

rand(1,N) plot for N=1000

The data in the plot displayed is completely random and

distributed between values of 0 and 1. To gather more information about the

random process in question, it’s statistical properties must be investigated.

Those properties are its mean, standard deviation and probability density

function (pdf).

Below are comparisons between the equations of the

theoretical mean and the sample mean.

Theoretical Mean

Sample Mean

Using the Matlab function mean(x) multiple times, various values

for the mean were returned – 0.5130, 0.5148, 0.492 etc. Since the values of the

mean returned all fluctuate in an even manor around the theoretical mean, it

can be said that the estimator of the sample mean described above is unbiased.

If the number of samples, N, is large enough then the sample mean will converge

to the theoretical mean. This is seen as N->, the sample mean equation can be written

as which translates to the integral of the of the

theoretical mean. The sample means accuracy is therefore proportional to the

sample size, N.

Theoretical Standard Deviation

= = ? 0.289

Sample Standard Deviation

Using the Matlab function std(x),

it was found that the standard deviation varied between each trial and

fluctuated around the theoretical standard deviation in an unbiased manner.

Just like the sample mean, the accuracy of the sample standard deviation was

proportional to the sample size, N.

Estimator Bias

To further probe into the statistical properties, the bias of the

two estimators was created and investigated by generating 10 realisations of X.

Figure

2: Sampled Mean and Standard Deviation Compared to the Theoretical Values

From looking at these graphs, it is shown that the sample mean

and sample standard deviation fluctuate above and below their theoretical

value. This facilitates and supports the notion that the estimators are

unbiased.

PDF Approximations

The first order statistics of our set of

random variables were inspected using the histogram()

function in Matlab.

a)

N = 100000, nbins =

1000

b)

N = 100000, nbins = 100

Figure 3: Comparison

Between the Approximxated PDFs and the Theoretical PDF

From the two histograms displayed it’s evident that the accuracy

is inversely proportional to the number of bins. This is so because for fewer

bins, each bin contains more samples and the accuracy is proportional the

number of samples present.

On the contrary, fewer bins reduces the plot’s resolution

significantly because each bin groups a greater number of x values onto the same

bar on the graph. Thus, the number of bins present is proportional to the

resolution. From this, a proposed optimal solution is to use a large sample

size and limit the number of bins used.

Normal Distribution

A normal distribution is a Gaussian distribution with zero mean

and a standard deviation equal to 1. It is described by the following pdf:

Estimator Bias of

Normal Distribution

Figure 4: Comparison of 2nd Order Statistics

Compared with their Theoretical Values

Just like the sample mean and sample

standard variation above, the normal distribution’s sample mean and sample

standard deviation also fluctuate in and around the theoretical value and are

unbiased estimators. a) N = 1000, bins =

100 b)

N = 1000, bins = 1000

Figure 5: Comparison between Approximated

PDF and Theoretical PDF

Approximated PDF

This figure shows clearly that the left-hand graph has too few

bins which affects its accuracy and resolution – although this is contrary to

what was said before, the accuracy of the right graph is less due to too few

bins resulting in quantisation error. The right figure has the same number of

bins as it does samples so they fit perfectly into the normal distribution.

1.2

Stochastic Processes

The Matlab code for each RP function was

run and the corresponding graphs were plotted together.

Figure 6: 2nd

Order Statistics of RP1, RP2 and RP3

In order to determine the Stationarity

of a random process it must be shown that its statistical properties are time

invariant. If the random processes second order statistics are time invariant

then this implies the random process is Wide Sense Stationary (WSS).

Referring to the above random processes; RP1’s ensemble mean is

linearly increasing as time increases. Time is represented with the sample

number. The ensemble standard deviation

shows time varying properties as well. Therefore, it is non-stationary.

Whereas RP2 and RP3 display time invariant statistics in their

ensemble statistics. They can therefore be labelled as stationary. RP3 shows

much greater fluctuation than RP2 from its steady state mean which is showcased

by its standard deviation being more than double that of RP3.

A WSS process can be described as ergodic if a single realisation

of adequate length is enough to deduce its stochastic properties. Thus, the

random process is said to be ergodic if the time average approaches the

ensemble average when evaluated as the following: .

Figure 7: 2nd Order Statistics of

RP1, RP2 and RP3

RPI is not ergodic because it is non-stationary. This can be

seen because the sample mean is not the same as the ensemble mean just as the

standard deviation doesn’t either. However, RP3 has sample statistics that

match its ensemble statistics significantly. It may be ergodic but this needs

to be confirmed by further mathematical analysis. RP2 has a significant amount

of fluctuation in its sample mean and standard deviation; significantly larger

than the other two processes. It can’t be concluded whether RP2 is non-ergodic

though due to too small of a sample size. Further mathematical analysis will

clarify this uncertainty.

Random Process 1

RP1 can be expressed as the following when the random variable

is defined as X ~ U(0,1);

The theoretical mean of V:

=

0.02n

And the standard deviation:

Both statistics depend on n, the sample number. Thus, RP1 is neither

stationary or ergodic.

Random Process 2

RP2 can be expressed as the following when the random variable

is defined as X ~ U(0,1);

M and A are both random variables constant within any given

realisation.

Theoretical Mean

Standard Deviation

Neither the mean nor standard deviation depend on the sample number

and therefore RP2 is stationary. However, for this random process the sample

mean settles at whatever random value is generated by A for all the

realisations and the value for A varies between each realisation and therefore

RP2 is not ergodic.

Random Process 3

RP3 can be expressed as the following when the random variable

is defined as X ~ U(0,1);

Both

a and m are constants.

Theoretical Mean

Standard Deviation

Once

again, the mean and standard deviation do not depend on the sample number and

thus RP3 is stationary just as RP2 is. However,