Type I and Type II Errors: Monte Carlo Simulations

We study in econometrics that a BLUE estimator avoids both Type I and Type II errors, today we will see what these errors are and how we can test this property of the estimator using Monte Carlo Simulations. First of all we will define the Type I error and Type II error.

Evidence:

 

H0 : He is Not a criminal

H1: He is a criminal

Differentiating between two possible outcomes depend upon the evidence generated, similarly in statistics and economics, differentiating between the competing hypothesis requires the criteria should be precise and clear enough.

Type I error:

Here type one error will be when we convict the person even tough he was innocent i.e. rejecting the true null hypothesis.

Type II error:

This is the error of deciding the person to be not criminal when in real this is not true i.e. Accepting the false null hypothesis.

There is a fine line between these two errors, if this hypothesis is from regression, then the precise slope coefficient will be necessary to avoid any of these errors.

Example:

Lets construct same example used in previous blogs,

population relation :

Y = 1 + 2X + u

where X and U are random and non related.

we estimate this model using

Y = α0 + α1 X

So here the hypothesis are :

H0 : α1 = 2

H1: α1 ≠ 2

Checking Type I error:

We have done 5000 simulations, with 5 percent confidence interval the test statistic should reject null only 250 times. And from our simulations the result it 248, which means we are not rejecting the true null hypothesis.

size of statistic

As it is a two tailed test, so each observation of the p value should be less than 2.5%. Suggesting that this statistic is accepting the true null more times than the specified criteria of 95%.  So this slope parameter is successfully avoiding the Type I error.

Type II error:

This simulation tests the slope coefficient against hypothesized marginally increasing values from 0 to 4. And it is expected that the P value of the test should be less or equal to 0.05 when it is hypothesized against true value of 2, other wise it should successfully reject the wrong null hypothesis.

power of statistic

This curve dips down to probability of 0.05 when it is tested against the true population coefficient, and it stays above 0.05 for all other observations. Hence we can confirm that there is no Type II error in this regression.

Discussion:

More over the primary feature of avoiding Type I and Type II error is mainly dependent upon the problems that regression can have like Multicollinearity, Hetroskedasticity and Auto-correlation, which we have checked in our assumptions and previous tests, as this OLS which is free of above mentioned problems, hence the tests were precise enough to differentiate between both null and alternative hypothesis. The purpose of introducing this is to use them as future references, when they are tested on the regression samples having artificially generated problems.

Stata do file: Type I and Type II error – example from simulations

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s