Estimates and Estimation

Post about Estimates and Estimation that include point estimation, interval estimation, level of significance, Properties of estimators, and real-life examples about estimate and estimation.

Estimating Mean using t Distribution

by

One can estimate the mean using the t distribution. In this post, we will discuss estimating mean using t distribution. The process of constructing a confidence interval using a t distribution is almost identical to that used to construct a confidence interval using the standard normal distribution.

First, we must know that variable $x$ is normally distributed with unknown standard deviation $\sigma$ and that we will draw a small sample ($n<30$). We then choose $c$, the desired level of confidence, and calculate the statistics $\overline{x}$ and $s$ from our sample group.

Margin of Error

The sample mean $\overline{x}$ will again be the best point estimate and the center of our interval. One can then calculate the margin of error for our estimate using the formula:

$$E=t_c \frac{s}{\sqrt{n}}$$

where $t_c$ is the critical t-value corresponding to the level of confidence $c$. The values of $t_c$ for common values of $c$ are given in the t-table. Make sure to use a degree of freedom of $n-1$.

Note that $t_c > z_c$ for the same value of $c$ since the t-distribution is wider, so we get a larger margin of error using the t-distribution.

Estimating mean using t distribution

Example: Estimating mean using t distribution

Suppose we had our sample of 5 women’s heights: 67, 63, 64, 65, 63. If it is known that women’s heights were normally distributed, but one does not know that $\sigma = 2.75$, then one can use the sample standard deviation $s$ as our estimate of $\sigma$, and then use a t-distribution interval.

The sample mean is $\overline{x} = 64.4$ inches and the sample standard deviation is $s=1.67$ inches. For a 95% confidence interval, the critical t-score for degrees of freedom is: $t_c=2.776$. So

\begin{align*}
E &= t_c \frac{s}{\sqrt{n}} \\
&=(2.776) \left(\frac{1.67}{\sqrt{5}}\right) \\
&=\approx 2.07
\end{align*}

So, our 95% confidence interval is

$$[64.4 – 2.07, 64.4 + 2.07] = [62.33, 66.47]$$

Exercise: Estimate Mean using t distribution

SAT Math scores are normally distributed. A sample of scores for 20 students has a sample mean of $\overline{x} = 522.8$ with a sample standard deviation of $s=154.5$.

  • Calculate the 90% confidence interval for the mean SAT Math Score.
  • Suppose the same sample mean and sample standard deviation had been obtained from a sample of size 16. What would the 90% confidence interval be?
  • Suppose the same sample mean and sample standard deviation were obtained from a sample of size 50. What would the 90% confidence interval be?

Assumptions using the t distribution

For this estimation to be valid, your data should meet one of the following conditions:

  1. The data is approximately normally distributed. This is the ideal scenario, especially for small sample sizes ($n < 30$).
  2. The sample size is large ($n \ge 30$). Thanks to the Central Limit Theorem, the sampling distribution of the mean will be approximately normal, even if the original population data is not. This makes the t-distribution robust for larger samples.

t-Distribution vs. Z-Distribution (Normal)

This is a common point of confusion. Here’s a simple decision guide:

Featuret-DistributionZ-Distribution (Normal)
Population $\sigma$UnknownKnown
Test Statistic$t=\dfrac{\overline{x} – \mu}{\frac{s}{\sqrt{n}}}$$t=\dfrac{\overline{x} – \mu}{\frac{\sigma}{\sqrt{n}}}$
VariabilityMore variable (thicker tails)
Shape Depends OnDegrees of Freedom (df)It is always the standard normal curve
When to UseMost real world situationsMost real-world situations

In practice, you will almost always use the t-distribution for estimating a population mean.

Finding the Critical t-Value

You can find critical t-values in several ways:

  1. t-Table (Statistical Table): The traditional method. You find the value at the intersection of your df row and your $\frac{\alpha}{2}$ column.
  2. Statistical Software: Programs like R, Python (with SciPy), SPSS, etc., can calculate it precisely.
  3. Calculators: Many advanced calculators (like the TI-84) have inverse t-functions.

For example, in Python, you would use scipy.stats.t.ppf(0.975, df=24) to get 2.064. (We use 0.975 because we need the cumulative probability up to the critical value, which is $1-\frac{\alpha}{2}$).

By following this process, you can reliably estimate a population mean even when you only have sample data, properly accounting for the uncertainty that comes with estimating the population standard deviation.

Data Analysis in R Language

Estimating the Mean

by

The mean is the first statistic we learn, the cornerstone of many analyses. But the question is how well do we understand its estimation? For statisticians, estimating the mean is more than just summing and dividing. It involves navigating assumptions, choosing appropriate methods, and understanding the implications of our choices. Let us delve deeper into the art and science of estimating the mean.

The Simple Sample Mean: A Foundation

The Formula of sample mean $\overline{x}= \frac{\sum\limits_{i=1}^n x_i}{n}$​​. The sample mean is the unbiased estimator of population mean ($\mu$) under ideal conditions (simple random sampling, independent and identically distributed data). Violating the assumption can lead to biased estimates. For large samples, the distribution of the sample mean approximates a normal distribution, regardless of the population distribution due to the Central Limit Theorem (CLT).

Weighted Means

Beyond Simple Random Sampling, For weighted means, observations have varying importance (e.g., survey data with different sampling weights). The formula of weighted mean is $ \overline{x}_w = \frac{\sum\limits_{i=1}^n w_ix_i}{\sum\limits_{i=1}^n w_i}$. Weighted means are used in Survey sampling, and dealing with non-response. In Stratified Sampling, estimate the mean when the population is divided into strata for getting reduced variance, and improved precision. In cluster sampling have unique challenges of estimating the mean with cluster sampling, where observations are grouped.

Robust Estimation

Robust Estimation is required when the sample mean is vulnerable to extreme values. The alternative of the sample mean is the median which emphasizes its robustness to outliers. The trimmed mean is also used to balance out the robustness and efficiency.

Confidence Intervals for Estimating the Mean

Confidence Intervals make use of standard error to estimate the mean to reflect the precision of the estimate. For small samples, t-distribution while for large samples, z-distribution is used for the construction of confidence intervals. Bootstrapping (a non-parametric method) can also be used for constructing confidence intervals, especially, useful when assumptions are violated.

Point Estimate: To estimate the population mean $\mu$ for a random variable $x$ using a sample of values, the best possible point estimate is the sample mean $\overline{x}$.

Interval Estimate: An interval estimate for mean $\mu$ is constructed by starting with sample mean $\overline{x}$ and adding a margin of error (S.E.) above and below the mean $\overline{x}$. The interval is of the form $(\overline{x} – SE, \overline{x} + SE)$.

Example: Suppose that the mean height of Pakistani men is between 67.5 and 70.5 inches with a level of confidence of $c = 0.90$. To estimate the men’s height, the sample mean $\overline{x}$ is 69 inches with a margin of error = 1.5 inches. That is, $(\overline{x} – SE, \overline{x}+SE) = (69 – 1.5, 69+1.5) = (67.5, 70.5)$.

Note that the margin of error used for constructing an interval estimate depends on the level of confidence interval. A larger level of confidence will result in a larger margin of error and hence a wider interval.

Estimating the Mean boxplot with mean

Calculating Margin of Error for a Large Sample Data

If a random variable $x$ is normally distributed (with a known population standard deviation $\sigma$) or if the sample size $n$ is at least 30 (we will apply Central Limit Theorem, which will guarantee that),

  • $\overline{x}$ is approximately normally distributed
  • $\mu_{\overline{x}} = \mu$
  • $\sigma_{\overline{x}}=\frac{\sigma}{\sqrt{n}}$

The mean value of $\overline{x}$ equals the estimated population mean $\mu$. Given the desired level of confidence $c$, it is try to find the amount of error $E$ necessary to ensure that the probability of $\overline{x}$ being within $E$ of the mean is $c$.

There are always two critical $Z$-scores ($\pm z_c$ which give the appropriate probability for the standard normal distribution), and the corresponding probability for the distribution of $\overline{x}$ is $z_c \times \sigma_{\overline{x}}$ or

$$E=z_c \frac{\sigma}{\sqrt{n}}$$

Usually, $\sigma$ is unknown, but if $n\ge 30$ then the sample standard deviation $s$ is generally a reasonable estimate.

Estimating the Mean Histogram

Dealing with Missing Data

When dealing with missing data, one can impute mean. Imputing the mean is simple but it can underestimate variance. One can also perform multiple imputations to account for the uncertainty.

Bayesian Estimation

In Bayesian estimation, the prior and posterior distributions are used for estimating the mean by incorporating prior information, updated beliefs about the mean, and handling uncertainty.

Summary

Estimating the mean is a fundamental statistical task, but it requires careful consideration of assumptions, data characteristics, and the goals of the analysis. By understanding the nuances of different estimation methods, statisticians can provide more accurate and reliable insights.

Exploratory Data Analysis in R Language

Consistency: A Property of Good Estimator

by

Consistency refers to the property of an estimator that as the sample size increases, the estimator converges in probability to the true value of the parameter being estimated. In other words, a consistent estimator will yield results that become more accurate and stable as more data points are collected.

Characteristics of a Consistent Estimator

A consistent has some important characteristics:

  • Convergence: The estimator will produce values that get closer to the true parameter value with larger samples.
  • Reliability: Provides reassurance that the estimates will be valid as more data is accounted for.

Examples of Consistent Estimators

  1. Sample Mean ($\overline{x}$): The sample mean is a consistent estimator of the population mean ($\mu$). A larger sample from a population converges to the actual population mean, compared to a smaller smaller.
  2. Sample Proportion ($\hat{p}$): The sample proportion is also a consistent estimator of the true population proportion. As the number of observations increases, the sample proportion gets closer to the true population proportion.

Question: $\hat{\theta}$ is a consistent estimator of the parameter $\theta$ of a given population if

  1. $\hat{\theta}$ is unbiased, and
  2. $var(\hat{\theta}) \rightarrow 0$ when $n\rightarrow \infty$

Answer: Suppose $X$ is random with mean $\mu$ and variance $\sigma^2$. If $X_1,X_2,\cdots,X_n$ is a random sample from $X$ then

\begin{align*}
E(\overline{X}) &= \mu\\
Var(\overline{X}) & = \frac{\sigma^2}{n}
\end{align*}

That is $\overline{X}$ is unbiased and $\lim\limits_{n\rightarrow\infty} Var(\overline{X}) = \lim\limits_{n\rightarrow\infty} \frac{\sigma^2}{n} =0$

Question: Show that the sample mean $\overline{X}$ of a random sample of size $n$ from the density function $f(x; \theta) = \frac{1}{\theta} e^{-\frac{x}{\theta}}, \qquad 0<x<\infty$ is a consistent estimator of the parameter $\theta$.

Answer: First, we need to check that $E(\overline{x})=\theta$, that is, the sample mean $\overline{X}$ is unbiased.

\begin{align*}
E(X) &= \mu = \int x\cdot f(x; \theta) dx = \int\limits_{0}^{\infty}x\cdot \frac{1}{\theta} e^{-\frac{x}{\theta}}dx\\
&= \frac{1}{\theta} \int\limits_{0}^{\infty} xe^{-\frac{x}{\theta}}dx\\
&= \frac{1}{\theta} \left[ \Big| -\theta x e^{-\frac{x}{\theta}}dx\Big|_{0}^{\infty} + \theta \int\limits_{0}^{\infty} e^{-\frac{x}{\theta}}dx \right]\\
&= \frac{1}{\theta} \left[0+\theta(-\theta) e^{-\frac{x}{\theta}}\big|_0^{\infty} \right] = \theta\\
E(X^2) &= \int x^2 f(x; \theta)dx = \int\limits_{0}^{\infty}x^2 \frac{1}{\theta} e^{-\frac{x}{\theta}}dx\\
&= \frac{1}{\theta}\left[ \Big| – x^2 \theta e^{-\frac{x}{\theta} }\Big|_{0}^{\infty} + \int\limits_0^\infty 2x\theta e^{-\frac{x}{\theta}}dx \right]\\
&= \frac{1}{\theta} \left[ 0 + 2\theta^2 \int\limits_0^\infty \frac{x}{\theta} e^{-\frac{x}{\theta}}dx\right]
\end{align*}

The expression is to be integrated into $E(X)$ which equals 0. Thus

\begin{align*}
E(X^2) &=\frac{1}{\theta} 2\theta^2\theta = 2\theta^2\\
Var(X) &=E(X^2) – [E(X)]^2 = 2\theta^2 – \theta^2 = \theta^2
and \quad Var(\overline{X}) &= \frac{\sigma^2}{n}\\
\lim\limits_{n\rightarrow \infty} \,\, Var(\overline{X}) &= \lim\limits_{n\rightarrow \infty} \frac{\sigma^2}{n} = 0
\end{align*}

Since $\overline{X}$ is unbiased and $Var(\overline{X})$ approaches 0 and $n\rightarrow \infty$, the $\overline{X}$ is a consistent estimator of $\theta$.

Importance of Consistency in Statistics

The following are a few key points about the importance of consistency in statistics:

Reliable Inferences: Consistent estimators ensure that as sample size increases, the estimates become closer and closer to the true population value/parameters. This helps researchers and statisticians to make sound inferences about a population based on sample data.

Foundation for Hypothesis Testing: Most of the statistical methods rely on consistent estimators. Consistency helps in validating the conclusions drawn from statistical tests, leading to confidence in decision-making.

Improved Accuracy: Since more data points are available due to the increase in sample size, the more consistently the estimates will converge to the true value. All this leads to more accurate statistical models, which can improve analysis and predictions.

Mitigating Sampling Error: Consistent estimators help to reduce the impact of random sampling error. As sample sizes increase, the variability in estimates tends to decrease, leading to more dependable conclusions.

Building Statistical Theory: Consistency is a fundamental concept in the development of statistical theory. It provides a rigorous foundation for designing and validating statistical methods and procedures.

Trust in Results: Consistency builds trust in the findings of statistical analyses. It is because the results are stable and reliable across different samples (due to large samples), therefore it is more likely to accept and act upon those results.

Framework for Model Development: In statistics and data science, developing models based on consistent estimators results in models with more accuracy.

Long-Term Decision Making: Consistency in data interpretation supports long-term planning, risk assessment, and resource allocation. It is required that businesses and organizations often make strategic decisions based on statistical analyses.

https://itfeature.com consistency a property of good estimator

R Frequently Asked Questions