.bg-main4.column[.vmiddle.content[
.amber[Aly Lamuri]  
Indonesia Medical Education and Research Institute
]]

---

---

.bg-main1.column[.vmiddle.content[
- .amber[Mean difference]
- One sample T-Test
- Unpaired T-Test
- Paired T-Test
- Effect size
]]

---

# Concept recall

`$$\bar{X} \xrightarrow{d} N(\mu, \frac{\sigma}{\sqrt{n}}) \tag{2}$$`
]

???

- Scaling and centering data following a normal distribution results in a
  standardized normal distribution (Z-distribution)
- Central limit theorem: sample mean from any distribution will follow
  normal distribution

???

Ideal: by knowing the parameter `$\mu$` and `$\sigma$`, we can directly compare our
sample mean to its corresponding population

???

Solution: use its statistics `$\bar{x}$` as an estimate

---

# Mean difference

---

.font2[
- Simply `$\bar{x} - \mu_0$`
- But there bound to be errors in our samples
- What do we do?
- An adjustment to .amber[standard error]
]

???

- One-sample mean difference test: compare mean from our data to previously
  reported mean in the population

---

`\begin{align}
SE &= \frac{\sigma}{\sqrt{n}} \tag{Standard Error} \\
\\
z  &= \frac{\bar{x} - \mu_0}{SE} \\
   &= \frac{\bar{x} - \mu_0}{^{\sigma}/\tiny{\sqrt{n}}} \tag{One-sample Test}
\end{align}`

???

- `$\mu_0$` is the expected (hypothesized) mean, often it is something we acquire
  from previous publications
- `$\sigma$` is the **known** standard deviation in **population**

<br> <br>
.font2[How do we get the p-value?]

(.amber[Hint:] Z-statistics follows the Z-distribution)

???

To get the p-value, we calculate acquired `$z$` statistics as a quantile of the
Z-distribution

---

# Example, please?

In a population of third-year electrical engineering students, we know the
.amber[average final score] of a particular course is .pink[70]. In measuring
students' comprehension, UKRIDA has established a standardized examination with
a .amber[standard deviation] of .pink[10]. We are interested to see whether
students registered to this year course have different average, where 18
students averagely scored 75 on the final exam.

]

---

---

.font2[
`\begin{align}
SE &= \frac{10}{\sqrt{18}} &= 2.36 \\
z  &= \frac{75 - 70}{2.36} &= 2.12
\end{align}`
]

---

# Where does it located in Z-distribution?

---

# Where is it relative to the significance value?

---

# What is the p-value?

.font2[
- Recall the hypothesis `$H_a: \bar{x} \neq \mu_0$`
- We need to conduct a two-tailed test to determine the p-value
- First we find the cummulative probability of `$z$` which satisfies:

`$$P(Z \leqslant 2.12\ |\ \mu,\sigma): Z \sim N(0, 1)$$`
]

```r
2 * {1 - pnorm(2.12, 0, 1)}
```

```
## [1] 0.034
```

---

# What do we learn?

.font2[
- Z-test requires the data to follow .amber[normal distribution]
- We do not need to know the parameter `$\mu \to$` We can hypothesize the value
- But we need the parameter `$\sigma$` to correctly compute `$z$`
]

---

# Any hidden message?

.font2[
- Yes!
- Z-test assumes normality
- So we need to perform goodness of fit or normality test
]

---

# Student's T-distribution

---

`\begin{align}
Let\ & X \sim t_\nu \tag{Notation} \\
P(X=x) &= \frac{\Gamma \big( \frac{\nu+1}{2} \big)}{\sqrt{\nu \pi}\ \Gamma \big( \frac{\nu}{2} \big)} \bigg( 1 + \frac{x^2}{\nu} \bigg) \tag{PDF} \\
\nu &= n - 1 \\
\\
Let\ & T \sim t_\nu \\
T &= Z \sqrt{\frac{\nu}{V}} \tag{Relationship}
\end{align}`

.font2[
- `$T:$` T-distribution with `$\nu$` degree of freedom
- `$Z:$` A standardized normal distribution
- `$V:$` A `$\chi^2$` distribution with `$\nu$` degree of freedom
]

???

- Importance: Use in parametric mean difference between two groups
- Student's T-distribution only depends on 1 parameter: `$\nu$` degree of freedom
- Degree of freedom is total subjects subtracted by 1

---

---

.bg-main1.column[.vmiddle.content[
- Mean difference
- .amber[One sample T-Test]
- Unpaired T-Test
- Paired T-Test
- Effect size
]]

---

# One sample T-Test

.font2[
- Similar to one sample Z-test
- Applied to data with unknown `$\sigma$`
- Use `$s$` as an estimate of the population parameter

`$$t = \frac{\bar{x}-\mu}{^s \big/ \tiny{\sqrt{n}}}$$`
]

---

# Example, please?

```r
set.seed(1)
x <- rnorm(20, mean=120, sd=20)
```

---

```r
summary(x)
```

```
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    75.7   112.3   127.2   123.8   135.2   151.9
```

```r
sd(x)
```

```
## [1] 18.3
```

---

.font2[
- We let `$x \sim N(120, 20)$`
- But our `$\bar{x}$` is 123.81
- With an `$s$` of 18.265
- And a `$\nu$` of 19
- Does our sample differ from the population?
]

---

## Formulate the hypothesis

---

## Determine the t-statistics

```r
t <- {{mean(x) - 120} / {sd(x) / sqrt(20)}} %T>% print()
```

```
## [1] 0.933
```

---

## Locate `$t$` statistics in the T-distribution

---

## P-value in a one-tailed test

```r
1 - pt(t, df=19)
```

```
## [1] 0.181
```

---

## P-value in a two-tailed test

```r
2 * {1 - pt(t, df=19)}
```

```
## [1] 0.363
```

---

# How is the result in `R`?

```r
t.test(x, mu=120)
```

```
## 
## 	One Sample t-test
## 
## data:  x
## t = 0.9, df = 19, p-value = 0.4
## alternative hypothesis: true mean is not equal to 120
## 95 percent confidence interval:
##  115 132
## sample estimates:
## mean of x 
##       124
```

## Conclusion

- Fail to reject `$H_0$`
- `$\bar{x} = \mu_0 = 120$`

---

.bg-main1.column[.vmiddle.content[
- Mean difference
- One sample T-Test
- .amber[Unpaired T-Test]
- Paired T-Test
- Effect size
]]

---

# Unpaired T-Test

.font2[
- T-Test conducted on two sample groups
- Aims to determine significance on the mean difference
- Types: .amber[Student's] and .amber[Welch's] T-Test
- Assumes .amber[normality] *or* a large sample size
- Also robust in non-normal data, .pink[*unless*] highly-skewed or have outliers
]

???

- Problem with robustness: we often use simulation with specified parameters
- But we don't know such parameters in real-world data
- It's more acceptable when we don't violate the normality assumption

.font2[
`\begin{align}
H_0 &: \bar{x}_1 - \bar{x}_2 = d \\
H_a &: \bar{x}_1 - \bar{x}_2 \neq d \\
d &= \mu_1 - \mu_2 = 0
\end{align}`
]

???

In a unique case, we may find `$d \neq 0$`

---

# Student's T-Test

.font2[
- T-Test with a pooled variance
- .amber[Requires:] Equal variance in two samples (tested with Levene's test)
- Uses pooled variance `$\to$` compute statistics and the degree of freedom
]

`\begin{align}
t &= \frac{\bar{x}_1 - \bar{x}_2 - d}{s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}} \tag{Statistics} \\
s_p &= \sqrt{\frac{(n_1 - 1) s_1^2 + (n_2 - 1) s_2^2}{\nu}} \tag{Pooled variance} \\
\nu &= n_1 + n_2 - 2 \tag{Degree of freedom}
\end{align}`

---

# Welch's T-Test

`\begin{align}
t &= \frac{\bar{x}_1 - \bar{x}_2 - d} {\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}} \tag{Statistics} \\
\nu &= \frac{(n_1-1)(n_2-1)}{(n_2-1)C^2 + (1-C^2)(n_1-1)} \tag{Degree of freedom} \\
\\
C &= \frac{\frac{s_1^2}{n_1}}{\frac{s_1^2}{n_1} \frac{s_2^2}{n_2}}
\end{align}`

.font2[
For the record:
- Still assumes .amber[normality]!
- Not-normally distributed data still accepted .pink[if] the sample size is .amber[large enough]
- How large is large enough? Re-check previous discussions on the .amber[CLT]
]

---

# Example, please?

---

.font2[
Suppose we are collecting data on body height. Our population of interest will
be students registered in UKRIDA, where we categorize sex as female and male.
We acquire a normally distributed data from both sexes, where:

- `$female \sim N(155, 15)$`
- `$male \sim N(170, 12)$`

We have a sample of 25 females and 30 males, and would like conduct a
hypothesis test on mean difference.
]

---

```r
set.seed(5)
tbl <- data.frame(
	"height" = c(rnorm(30, 170, 8), rnorm(25, 155, 16)),
	"sex" = c(rep("male", 30), rep("female", 25))
)

tapply(tbl$height, tbl$sex, summary)
```

```
## $female
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     123     146     154     158     173     190 
## 
## $male
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     152     165     168     170     177     184
```

```r
tapply(tbl$height, tbl$sex, sd)
```

```
## female   male 
##  17.98   7.93
```

---

???

Does it follow the normal distribution?

---

```r
tapply(tbl$height, tbl$sex, shapiro.test)
```

```
## $female
## 
## 	Shapiro-Wilk normality test
## 
## data:  X[[i]]
## W = 1, p-value = 0.7
## 
## 
## $male
## 
## 	Shapiro-Wilk normality test
## 
## data:  X[[i]]
## W = 1, p-value = 0.2
```

???

Yes, each group follows a normal distribution

---

```r
car::leveneTest(tbl$height ~ tbl$sex)
```

```
## Levene's Test for Homogeneity of Variance (center = median)
##       Df F value  Pr(>F)    
## group  1    14.3 0.00039 ***
##       53                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
```

---

```r
t.test(height ~ sex, data=tbl, var.equal=FALSE)
```

```
## 
## 	Welch Two Sample t-test
## 
## data:  height by sex
## t = -3, df = 32, p-value = 0.004
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -19.87  -4.07
## sample estimates:
## mean in group female   mean in group male 
##                  158                  170
```

---

```r
t.test(height ~ sex, data=tbl, var.equal=TRUE)
```

```
## 
## 	Two Sample t-test
## 
## data:  height by sex
## t = -3, df = 53, p-value = 0.002
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -19.27  -4.67
## sample estimates:
## mean in group female   mean in group male 
##                  158                  170
```

???

- The p-value is *much lower* compared to Welch's T-Test
- A low p-value does not mean a bad outcome per se, but we need to be cautious
- Especially when we have violation on required assumptions
- Always be wary of statistical error!

---

???

- Visualizing your findings is important
- Both figures give similar information, but conveyed in different fashions

---

.bg-main1.column[.vmiddle.content[
- Mean difference
- One sample T-Test
- Unpaired T-Test
- .amber[Paired T-Test]
- Effect size
]]

---

# Paired T-Test

.font2[
- Previous analysis implicitly assumes .pink[independence]
- Some observations may violate such an assumption
- Cases:
  - Difference between multiple measurements
  - Probability events where each instance influence another
]

---

# Measuring mean differences

---

.font2[
- Two-sample T-Test does not take into account difference within subjects
- In fact, it is a violation of its stringent assumption
- To measure difference within same subjects, we need to reduce the complexity
]

---

.font2[
- Suppose `$\mu_1$` and `$\mu_2$` represent measurement in `$t_1$` and `$t_2$`
- Both measures represent same subject (within comparison)
- We can calculate the difference between both samples:

`$$\mu_d = \mu_1 - \mu_2$$`
]

---

`\begin{align}
H_0 &: \mu_d = 0 \\
H_a &: \mu_d \neq 0
\end{align}`
]

---

# Example, please?

---

In the current investigation, we are looking for the effect of a certain
anti-hipertensive drug. First we measure the blood pressure baseline, then
prescribe the drug to all subjects. Then, we re-measure the blood pressure
after one month. Each subject has a unique identifier, so we can specify mean
differences within paired samples. Suppose we have the following scenario in 30
sampled subjects:

- `$X_1 \sim N(140, 12)$`
- `$X_2 \sim N(130, 17)$`

]

---

.font2.center[
Set our hypotheses:

`\begin{align}
H_0 &: \bar{x}_d = 0 \\
H_a &: \bar{x}_d \neq 0
\end{align}`
]

---

```r
set.seed(1)
tbl <- data.frame(
	"bp" = c(rnorm(30, 140, 12), rnorm(30, 133, 17)),
	"time" = c(rep("Before", 30), rep("After", 30)) %>%
		factor(levels=c("Before", "After"))
)

# Measure the mean of mean difference
md <- with(tbl, bp[time=="Before"] - bp[time=="After"])

# Calculate t-statistics
t <- {mean(md)} / {sd(md) / sqrt(30)} %T>% print()
```

```
## [1] 3.12
```

```r
# Obtain p-value for a two-sided test
2 * {1 - pt(t, df=29)}
```

```
## [1] 0.076
```

---

```r
# Comparison to built-in one-sample T-Test
t.test(md, mu=0)
```

```
## 
## 	One Sample t-test
## 
## data:  md
## t = 2, df = 29, p-value = 0.08
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.64 12.10
## sample estimates:
## mean of x 
##      5.73
```

---

```r
# Comparison to built-in paired T-Test
t.test(bp ~ time, data=tbl, paired=TRUE)
```

```
## 
## 	Paired t-test
## 
## data:  bp by time
## t = 2, df = 29, p-value = 0.08
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.64 12.10
## sample estimates:
## mean of the differences 
##                    5.73
```

---

# Choosing an .amber[appropriate] statistical test

.font2[
- One-sample test:
  - Known `$\sigma \to$` use Z-Test
  - Unknown `$\sigma \to$` use one-sample T-Test
]

.font2[
- Two-sample test `$\to$` do Levene's test
  - Equal variance: Student's method (pooled variance)
  - Unequal variance: Welch's method
]

---

.bg-main1.column[.vmiddle.content[
- Mean difference
- One sample T-Test
- Unpaired T-Test
- Paired T-Test
- .amber[Effect size]
]]

---

# Effect size

`\begin{align}
d &= \frac{\bar{x}_1 - \bar{x}_2}{s_p} \tag{Cohen's D} \\
s_p &= \sqrt{\frac{(s_1^2 + s_2^2)}{2}} \tag{Pooled SD}
\end{align}`

---

# Example, please?

---

```r
set.seed(1)
tbl <- data.frame(
	"bp" = c(rnorm(30, 140, 12), rnorm(30, 133, 17)),
	"time" = c(rep("Before", 30), rep("After", 30)) %>%
		factor(levels=c("Before", "After"))
)

# Measure the mean of mean difference
md <- with(tbl, bp[time=="Before"] - bp[time=="After"])

# Calculate t-statistics
t <- {mean(md)} / {sd(md) / sqrt(30)} %T>% print()
```

```
## [1] 3.12
```

```r
# Obtain p-value for a two-sided test
2 * {1 - pt(t, df=29)}
```

```
## [1] 0.076
```

---

```r
# Calculate pooled standard deviation
sp <- sqrt({with(tbl,
	tapply(bp, time, var, simplify=FALSE)) %>% {do.call(add, .)}
} / 2) %T>% print()
```

```
## [1] 12.4
```

```r
# Measure Cohen's distance
{with(tbl,
	tapply(bp, time, mean, simplify=FALSE)) %>% {do.call(subtract, .)}
} / sp
```

```
## [1] 0.464
```

---

```r
# Calculate power using the `psych` package
d <- psych::cohen.d(tbl ~ time) %T>% print()
```

```
## Call: psych::cohen.d(x = tbl ~ time)
## Cohen d statistic of difference between two means
##    lower effect upper
## bp -0.99  -0.47  0.05
## 
## Multivariate (Mahalanobis) distance between groups
## [1] 0.47
## r equivalent of difference between two means
##    bp 
## -0.23
```

---

```r
# Power analysis using previous information
pwr::pwr.t.test(n=30, d=d$cohen.d[[2]], sig.level=0.05, type="paired")
```

```
## 
##      Paired t test power calculation 
## 
##               n = 30
##               d = 0.472
##       sig.level = 0.05
##           power = 0.704
##     alternative = two.sided
## 
## NOTE: n is number of *pairs*
```

---

Notes for current slide

Notes for next slide

Parametric: Mean in Two Groups

Aly Lamuri
Indonesia Medical Education and Research Institute

1 / 26

Overview

Mean difference
One sample T-Test
Unpaired T-Test
Paired T-Test
Effect size

1 / 26

Concept recall

$\begin{matrix} (1) & \frac{x - \bar{x}}{s} \sim N (0, 1) \end{matrix}$

$\begin{matrix} (2) & \bar{X} \overset{d}{\to} N (μ, \frac{σ}{\sqrt{n}}) \end{matrix}$

1 / 26

Scaling and centering data following a normal distribution results in a standardized normal distribution (Z-distribution)
Central limit theorem: sample mean from any distribution will follow normal distribution

Concept recall

$\begin{matrix} (1) & \frac{x - \bar{x}}{s} \sim N (0, 1) \end{matrix}$

$\begin{matrix} (2) & \bar{X} \overset{d}{\to} N (μ, \frac{σ}{\sqrt{n}}) \end{matrix}$

With known $μ$ and $σ$ , we can make a direct comparison

1 / 26

Scaling and centering data following a normal distribution results in a standardized normal distribution (Z-distribution)
Central limit theorem: sample mean from any distribution will follow normal distribution

Ideal: by knowing the parameter $μ$ and $σ$ , we can directly compare our sample mean to its corresponding population

Concept recall

$\begin{matrix} (1) & \frac{x - \bar{x}}{s} \sim N (0, 1) \end{matrix}$

$\begin{matrix} (2) & \bar{X} \overset{d}{\to} N (μ, \frac{σ}{\sqrt{n}}) \end{matrix}$

With known $μ$ and $σ$ , we can make a direct comparison

But...

1 / 26

Scaling and centering data following a normal distribution results in a standardized normal distribution (Z-distribution)
Central limit theorem: sample mean from any distribution will follow normal distribution

Ideal: by knowing the parameter $μ$ and $σ$ , we can directly compare our sample mean to its corresponding population

Concept recall

$\begin{matrix} (1) & \frac{x - \bar{x}}{s} \sim N (0, 1) \end{matrix}$

$\begin{matrix} (2) & \bar{X} \overset{d}{\to} N (μ, \frac{σ}{\sqrt{n}}) \end{matrix}$

With known $μ$ and $σ$ , we can make a direct comparison

But...

How if we don't know $μ$ ?

1 / 26

Scaling and centering data following a normal distribution results in a standardized normal distribution (Z-distribution)
Central limit theorem: sample mean from any distribution will follow normal distribution

Ideal: by knowing the parameter $μ$ and $σ$ , we can directly compare our sample mean to its corresponding population

Solution: use its statistics $\bar{x}$ as an estimate

Mean differenceSimply ¯x−μ0x¯−μ0
But there bound to be errors in our samples
What do we do?
An adjustment to standard error

2 / 26

One-sample mean difference test: compare mean from our data to previously reported mean in the population

Mean difference

$\begin{aligned} (Standard Error) & S E & = \frac{σ}{\sqrt{n}} \\ z & = \frac{\bar{x} - μ_{0}}{S E} \\ (One-sample Test) & = \frac{\bar{x} - μ_{0}}{^{σ} / \sqrt{n}} \end{aligned}$

2 / 26

$μ_{0}$ is the expected (hypothesized) mean, often it is something we acquire from previous publications
$σ$ is the known standard deviation in population

Mean difference

$\begin{aligned} (Standard Error) & S E & = \frac{σ}{\sqrt{n}} \\ z & = \frac{\bar{x} - μ_{0}}{S E} \\ (One-sample Test) & = \frac{\bar{x} - μ_{0}}{^{σ} / \sqrt{n}} \end{aligned}$

How do we get the p-value?

2 / 26

$μ_{0}$ is the expected (hypothesized) mean, often it is something we acquire from previous publications
$σ$ is the known standard deviation in population

Mean difference

$\begin{aligned} (Standard Error) & S E & = \frac{σ}{\sqrt{n}} \\ z & = \frac{\bar{x} - μ_{0}}{S E} \\ (One-sample Test) & = \frac{\bar{x} - μ_{0}}{^{σ} / \sqrt{n}} \end{aligned}$

How do we get the p-value?

(Hint: Z-statistics follows the Z-distribution)

2 / 26

$μ_{0}$ is the expected (hypothesized) mean, often it is something we acquire from previous publications
$σ$ is the known standard deviation in population

To get the p-value, we calculate acquired $z$ statistics as a quantile of the Z-distribution

Example, please?

In a population of third-year electrical engineering students, we know the average final score of a particular course is 70. In measuring students' comprehension, UKRIDA has established a standardized examination with a standard deviation of 10. We are interested to see whether students registered to this year course have different average, where 18 students averagely scored 75 on the final exam.

$\begin{aligned} H_{0} & : \bar{x} = μ_{0} \\ H_{a} & : \bar{x} \neq μ_{0} \end{aligned}$

3 / 26

Example, please?

$\begin{aligned} S E & = \frac{10}{\sqrt{18}} & = 2.36 \\ z & = \frac{75 - 70}{2.36} & = 2.12 \end{aligned}$

3 / 26

Where does it located in Z-distribution?

3 / 26

Where is it relative to the significance value?

3 / 26

What is the p-value?

Recall the hypothesis $H_{a} : \bar{x} \neq μ_{0}$
We need to conduct a two-tailed test to determine the p-value
First we find the cummulative probability of $z$ which satisfies:

$P (Z ⩽ 2.12 | μ, σ) : Z \sim N (0, 1)$

3 / 26

What is the p-value?

Recall the hypothesis $H_{a} : \bar{x} \neq μ_{0}$
We need to conduct a two-tailed test to determine the p-value
First we find the cummulative probability of $z$ which satisfies:

$P (Z ⩽ 2.12 | μ, σ) : Z \sim N (0, 1)$

Subtract $P (Z = z)$ from 1
Multiply by 2

3 / 26

What is the p-value?

Recall the hypothesis $H_{a} : \bar{x} \neq μ_{0}$
We need to conduct a two-tailed test to determine the p-value
First we find the cummulative probability of $z$ which satisfies:

$P (Z ⩽ 2.12 | μ, σ) : Z \sim N (0, 1)$

Subtract $P (Z = z)$ from 1
Multiply by 2

2 * {1 - pnorm(2.12, 0, 1)}

## [1] 0.034

3 / 26

What do we learn?Z-test requires the data to follow normal distribution
We do not need to know the parameter μ→μ→ We can hypothesize the value
But we need the parameter σσ to correctly compute zz

4 / 26

Any hidden message?Yes!
Z-test assumes normality
So we need to perform goodness of fit or normality test

4 / 26

Any hidden message?

Yes!
Z-test assumes normality
So we need to perform goodness of fit or normality test

What if we do not know $σ$ ?

4 / 26

Any hidden message?

Yes!
Z-test assumes normality
So we need to perform goodness of fit or normality test

What if we do not know $σ$ ?

We are unable to use the Z-distribution

4 / 26

Any hidden message?

Yes!
Z-test assumes normality
So we need to perform goodness of fit or normality test

What if we do not know $σ$ ?

We are unable to use the Z-distribution

Solution: use Student's T-distribution

4 / 26

Student's T-distribution

$\begin{aligned} (Notation) & L e t & X \sim t_{ν} \\ (PDF) & P (X = x) & = \frac{Γ (\frac{ν + 1}{2})}{\sqrt{ν π} Γ (\frac{ν}{2})} (1 + \frac{x^{2}}{ν}) \\ ν & = n - 1 \\ L e t & T \sim t_{ν} \\ (Relationship) & T & = Z \sqrt{\frac{ν}{V}} \end{aligned}$

5 / 26

Student's T-distribution

$T :$ T-distribution with $ν$ degree of freedom
$Z :$ A standardized normal distribution
$V :$ A $χ^{2}$ distribution with $ν$ degree of freedom

5 / 26

Importance: Use in parametric mean difference between two groups
Student's T-distribution only depends on 1 parameter: $ν$ degree of freedom
Degree of freedom is total subjects subtracted by 1

Student's T-distribution

5 / 26

Overview

Mean difference
One sample T-Test
Unpaired T-Test
Paired T-Test
Effect size

6 / 26

One sample T-Test

Similar to one sample Z-test
Applied to data with unknown $σ$
Use $s$ as an estimate of the population parameter

$t = \frac{\bar{x} - μ}{^{s} / \sqrt{n}}$

7 / 26

Example, please?

set.seed(1)
x <- rnorm(20, mean=120, sd=20)

summary(x)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    75.7   112.3   127.2   123.8   135.2   151.9

sd(x)

## [1] 18.3

8 / 26

Example, please?

set.seed(1)
x <- rnorm(20, mean=120, sd=20)

We let $x \sim N (120, 20)$
But our $\bar{x}$ is 123.81
With an $s$ of 18.265
And a $ν$ of 19
Does our sample differ from the population?

8 / 26

Example, please?

set.seed(1)
x <- rnorm(20, mean=120, sd=20)

Formulate the hypothesis

$\begin{aligned} H_{0} & : \bar{x} = 120 \\ H_{a} & : \bar{x} \neq 120 \end{aligned}$

8 / 26

Example, please?

set.seed(1)
x <- rnorm(20, mean=120, sd=20)

Determine the t-statistics

$t = \frac{\bar{x} - μ}{^{s} / \sqrt{n}}$

t <- {{mean(x) - 120} / {sd(x) / sqrt(20)}} %T>% print()

## [1] 0.933

8 / 26

Locate $t$ statistics in the T-distribution

8 / 26

P-value in a one-tailed test

1 - pt(t, df=19)

## [1] 0.181

8 / 26

P-value in a two-tailed test

2 * {1 - pt(t, df=19)}

## [1] 0.363

8 / 26

How is the result in `R`?

t.test(x, mu=120)

## 
##     One Sample t-test
## 
## data:  x
## t = 0.9, df = 19, p-value = 0.4
## alternative hypothesis: true mean is not equal to 120
## 95 percent confidence interval:
##  115 132
## sample estimates:
## mean of x 
##       124

8 / 26

How is the result in `R`?

t.test(x, mu=120)

## 
##     One Sample t-test
## 
## data:  x
## t = 0.9, df = 19, p-value = 0.4
## alternative hypothesis: true mean is not equal to 120
## 95 percent confidence interval:
##  115 132
## sample estimates:
## mean of x 
##       124

Conclusion

Fail to reject $H_{0}$
$\bar{x} = μ_{0} = 120$

8 / 26

Overview

Mean difference
One sample T-Test
Unpaired T-Test
Paired T-Test
Effect size

9 / 26

Unpaired T-TestT-Test conducted on two sample groups
Aims to determine significance on the mean difference
Types: Student's and Welch's T-Test
Assumes normality or a large sample size
Also robust in non-normal data, unless highly-skewed or have outliers

10 / 26

Problem with robustness: we often use simulation with specified parameters
But we don't know such parameters in real-world data
It's more acceptable when we don't violate the normality assumption

Unpaired T-Test

T-Test conducted on two sample groups
Aims to determine significance on the mean difference
Types: Student's and Welch's T-Test
Assumes normality or a large sample size
Also robust in non-normal data, unless highly-skewed or have outliers

$\begin{aligned} H_{0} & : {\bar{x}}_{1} - {\bar{x}}_{2} = d \\ H_{a} & : {\bar{x}}_{1} - {\bar{x}}_{2} \neq d \\ d & = μ_{1} - μ_{2} = 0 \end{aligned}$

10 / 26

Problem with robustness: we often use simulation with specified parameters
But we don't know such parameters in real-world data
It's more acceptable when we don't violate the normality assumption

In a unique case, we may find $d \neq 0$

Student's T-TestT-Test with a pooled variance
Requires: Equal variance in two samples (tested with Levene's test)
Uses pooled variance →→ compute statistics and the degree of freedom

11 / 26

Student's T-Test

T-Test with a pooled variance
Requires: Equal variance in two samples (tested with Levene's test)
Uses pooled variance $\to$ compute statistics and the degree of freedom

$\begin{aligned} (Statistics) & t & = \frac{{\bar{x}}_{1} - {\bar{x}}_{2} - d}{s_{p} \sqrt{\frac{1}{n_{1}} + \frac{1}{n_{2}}}} \\ (Pooled variance) & s_{p} & = \sqrt{\frac{(n_{1} - 1) s_{1}^{2} + (n_{2} - 1) s_{2}^{2}}{ν}} \\ (Degree of freedom) & ν & = n_{1} + n_{2} - 2 \end{aligned}$

11 / 26

Student's T-Test

T-Test with a pooled variance
Requires: Equal variance in two samples (tested with Levene's test)
Uses pooled variance $\to$ compute statistics and the degree of freedom

Often, our data violate the equal variance assumption

Solution: Welch's T-Test

11 / 26

Welch's T-Test

$\begin{aligned} (Statistics) & t & = \frac{{\bar{x}}_{1} - {\bar{x}}_{2} - d}{\sqrt{\frac{s_{1}^{2}}{n_{1}} + \frac{s_{2}^{2}}{n_{2}}}} \\ (Degree of freedom) & ν & = \frac{(n_{1} - 1) (n_{2} - 1)}{(n_{2} - 1) C^{2} + (1 - C^{2}) (n_{1} - 1)} \\ C & = \frac{\frac{s_{1}^{2}}{n_{1}}}{\frac{s_{1}^{2}}{n_{1}} \frac{s_{2}^{2}}{n_{2}}} \end{aligned}$

12 / 26

Welch's T-Test

For the record:

Still assumes normality!
Not-normally distributed data still accepted if the sample size is large enough
How large is large enough? Re-check previous discussions on the CLT

12 / 26

Example, please?

Suppose we are collecting data on body height. Our population of interest will be students registered in UKRIDA, where we categorize sex as female and male. We acquire a normally distributed data from both sexes, where:

$f e m a l e \sim N (155, 15)$
$m a l e \sim N (170, 12)$

We have a sample of 25 females and 30 males, and would like conduct a hypothesis test on mean difference.

13 / 26

Example, please?

set.seed(5)
tbl <- data.frame(
    "height" = c(rnorm(30, 170, 8), rnorm(25, 155, 16)),
    "sex" = c(rep("male", 30), rep("female", 25))
)
tapply(tbl$height, tbl$sex, summary)

## $female
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     123     146     154     158     173     190 
## 
## $male
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##     152     165     168     170     177     184

tapply(tbl$height, tbl$sex, sd)

## female   male 
##  17.98   7.93

13 / 26

Example, please?

13 / 26

Does it follow the normal distribution?

Example, please?

tapply(tbl$height, tbl$sex, shapiro.test)

## $female
## 
##     Shapiro-Wilk normality test
## 
## data:  X[[i]]
## W = 1, p-value = 0.7
## 
## 
## $male
## 
##     Shapiro-Wilk normality test
## 
## data:  X[[i]]
## W = 1, p-value = 0.2

13 / 26

Yes, each group follows a normal distribution

Example, please?

car::leveneTest(tbl$height ~ tbl$sex)

## Levene's Test for Homogeneity of Variance (center = median)
##       Df F value  Pr(>F)    
## group  1    14.3 0.00039 ***
##       53                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

14 / 26

Example, please?

car::leveneTest(tbl$height ~ tbl$sex)

## Levene's Test for Homogeneity of Variance (center = median)
##       Df F value  Pr(>F)    
## group  1    14.3 0.00039 ***
##       53                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Levene's test suggests heterogenous variance (hint: significant p-value)

14 / 26

Example, please?

t.test(height ~ sex, data=tbl, var.equal=FALSE)

## 
##     Welch Two Sample t-test
## 
## data:  height by sex
## t = -3, df = 32, p-value = 0.004
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -19.87  -4.07
## sample estimates:
## mean in group female   mean in group male 
##                  158                  170

Perform Welch's T-Test since sampled variances are not equal

15 / 26

Example, please?

t.test(height ~ sex, data=tbl, var.equal=TRUE)

## 
##     Two Sample t-test
## 
## data:  height by sex
## t = -3, df = 53, p-value = 0.002
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -19.27  -4.67
## sample estimates:
## mean in group female   mean in group male 
##                  158                  170

Student's T-Test, to demonstrate type-I error inflation (hint: look at the p-value)

16 / 26

The p-value is much lower compared to Welch's T-Test
A low p-value does not mean a bad outcome per se, but we need to be cautious
Especially when we have violation on required assumptions
Always be wary of statistical error!

Example, please?

17 / 26

Visualizing your findings is important
Both figures give similar information, but conveyed in different fashions

Overview

Mean difference
One sample T-Test
Unpaired T-Test
Paired T-Test
Effect size

18 / 26

Paired T-TestPrevious analysis implicitly assumes independence
Some observations may violate such an assumption
Cases:Difference between multiple measurements
Probability events where each instance influence another


19 / 26

Measuring mean differencesTwo-sample T-Test does not take into account difference within subjects
In fact, it is a violation of its stringent assumption
To measure difference within same subjects, we need to reduce the complexity

20 / 26

Measuring mean differences

Suppose $μ_{1}$ and $μ_{2}$ represent measurement in $t_{1}$ and $t_{2}$
Both measures represent same subject (within comparison)
We can calculate the difference between both samples:

$μ_{d} = μ_{1} - μ_{2}$

20 / 26

Measuring mean differences

Now we only need to take into account one sample differences
We hypothesize:

$\begin{aligned} H_{0} & : μ_{d} = 0 \\ H_{a} & : μ_{d} \neq 0 \end{aligned}$

20 / 26

Measuring mean differences

Now we only need to take into account one sample differences
We hypothesize:

$\begin{aligned} H_{0} & : μ_{d} = 0 \\ H_{a} & : μ_{d} \neq 0 \end{aligned}$

Sounds familiar?
Because it is!
We can apply one-sample T-Test to $μ_{d}$

20 / 26

Example, please?

In the current investigation, we are looking for the effect of a certain anti-hipertensive drug. First we measure the blood pressure baseline, then prescribe the drug to all subjects. Then, we re-measure the blood pressure after one month. Each subject has a unique identifier, so we can specify mean differences within paired samples. Suppose we have the following scenario in 30 sampled subjects:

$X_{1} \sim N (140, 12)$
$X_{2} \sim N (130, 17)$

21 / 26

Example, please?

Set our hypotheses:

$\begin{aligned} H_{0} & : {\bar{x}}_{d} = 0 \\ H_{a} & : {\bar{x}}_{d} \neq 0 \end{aligned}$

21 / 26

Example, please?

set.seed(1)
tbl <- data.frame(
    "bp" = c(rnorm(30, 140, 12), rnorm(30, 133, 17)),
    "time" = c(rep("Before", 30), rep("After", 30)) %>%
        factor(levels=c("Before", "After"))
)
# Measure the mean of mean difference
md <- with(tbl, bp[time=="Before"] - bp[time=="After"])
# Calculate t-statistics
t <- {mean(md)} / {sd(md) / sqrt(30)} %T>% print()

## [1] 3.12

# Obtain p-value for a two-sided test
2 * {1 - pt(t, df=29)}

## [1] 0.076

21 / 26

Example, please?

# Comparison to built-in one-sample T-Test
t.test(md, mu=0)

## 
##     One Sample t-test
## 
## data:  md
## t = 2, df = 29, p-value = 0.08
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.64 12.10
## sample estimates:
## mean of x 
##      5.73

21 / 26

Example, please?

# Comparison to built-in paired T-Test
t.test(bp ~ time, data=tbl, paired=TRUE)

## 
##     Paired t-test
## 
## data:  bp by time
## t = 2, df = 29, p-value = 0.08
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.64 12.10
## sample estimates:
## mean of the differences 
##                    5.73

21 / 26

Choosing an appropriate statistical testBoth T-Test and Z-Test assume normality

22 / 26

Choosing an appropriate statistical testBoth T-Test and Z-Test assume normality

One-sample test:Known σ→σ→ use Z-Test
Unknown σ→σ→ use one-sample T-Test

22 / 26

Choosing an appropriate statistical testBoth T-Test and Z-Test assume normality

One-sample test:Known σ→σ→ use Z-Test
Unknown σ→σ→ use one-sample T-Test

Two-sample test →→ do Levene's testEqual variance: Student's method (pooled variance)
Unequal variance: Welch's method

22 / 26

Choosing an appropriate statistical testBoth T-Test and Z-Test assume normality

One-sample test:Known σ→σ→ use Z-Test
Unknown σ→σ→ use one-sample T-Test

Two-sample test →→ do Levene's testEqual variance: Student's method (pooled variance)
Unequal variance: Welch's method

Paired T-Test: Basically one-sample T-Test on sampled differences

22 / 26

Overview

Mean difference
One sample T-Test
Unpaired T-Test
Paired T-Test
Effect size

23 / 26

Effect size

Concept recall: statistical power
Related topics: sample size and significance level

$\begin{aligned} (Cohen's D) & d & = \frac{{\bar{x}}_{1} - {\bar{x}}_{2}}{s_{p}} \\ (Pooled SD) & s_{p} & = \sqrt{\frac{(s_{1}^{2} + s_{2}^{2})}{2}} \end{aligned}$

24 / 26

Example, please?

set.seed(1)
tbl <- data.frame(
    "bp" = c(rnorm(30, 140, 12), rnorm(30, 133, 17)),
    "time" = c(rep("Before", 30), rep("After", 30)) %>%
        factor(levels=c("Before", "After"))
)
# Measure the mean of mean difference
md <- with(tbl, bp[time=="Before"] - bp[time=="After"])
# Calculate t-statistics
t <- {mean(md)} / {sd(md) / sqrt(30)} %T>% print()

## [1] 3.12

# Obtain p-value for a two-sided test
2 * {1 - pt(t, df=29)}

## [1] 0.076

25 / 26

Example, please?

# Calculate pooled standard deviation
sp <- sqrt({with(tbl,
    tapply(bp, time, var, simplify=FALSE)) %>% {do.call(add, .)}
} / 2) %T>% print()

## [1] 12.4

# Measure Cohen's distance
{with(tbl,
    tapply(bp, time, mean, simplify=FALSE)) %>% {do.call(subtract, .)}
} / sp

## [1] 0.464

25 / 26

Example, please?

# Calculate power using the `psych` package
d <- psych::cohen.d(tbl ~ time) %T>% print()

## Call: psych::cohen.d(x = tbl ~ time)
## Cohen d statistic of difference between two means
##    lower effect upper
## bp -0.99  -0.47  0.05
## 
## Multivariate (Mahalanobis) distance between groups
## [1] 0.47
## r equivalent of difference between two means
##    bp 
## -0.23

25 / 26

Example, please?

# Power analysis using previous information
pwr::pwr.t.test(n=30, d=d$cohen.d[[2]], sig.level=0.05, type="paired")

## 
##      Paired t test power calculation 
## 
##               n = 30
##               d = 0.472
##       sig.level = 0.05
##           power = 0.704
##     alternative = two.sided
## 
## NOTE: n is number of *pairs*

26 / 26

Query?

26 / 26

Overview

Mean difference
One sample T-Test
Unpaired T-Test
Paired T-Test
Effect size

1 / 26

Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
Number + Return	Go to specific slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help

Esc	Back to slideshow

Concept recall

Concept recall

Concept recall

Concept recall

Mean difference

Mean difference

Mean difference

Mean difference

Example, please?

Example, please?

Where does it located in Z-distribution?

Where is it relative to the significance value?

What is the p-value?

What is the p-value?

What is the p-value?

What do we learn?

Any hidden message?

Any hidden message?

Any hidden message?

Any hidden message?

Student's T-distribution

Student's T-distribution

Student's T-distribution

One sample T-Test

Example, please?

Example, please?

Example, please?

Formulate the hypothesis

Example, please?

Determine the t-statistics

Locate tt statistics in the T-distribution

P-value in a one-tailed test

P-value in a two-tailed test

How is the result in R?

How is the result in R?

Conclusion

Unpaired T-Test

Unpaired T-Test

Student's T-Test

Student's T-Test

Student's T-Test

Welch's T-Test

Welch's T-Test

Example, please?

Example, please?

Example, please?

Example, please?

Example, please?

Example, please?

Example, please?

Example, please?

Example, please?

Paired T-Test

Measuring mean differences

Measuring mean differences

Measuring mean differences

Measuring mean differences

Example, please?

Example, please?

Example, please?

Example, please?

Example, please?

Choosing an appropriate statistical test

Choosing an appropriate statistical test

Choosing an appropriate statistical test

Choosing an appropriate statistical test

Effect size

Example, please?

Example, please?

Example, please?

Example, please?

Help

Locate $t$ statistics in the T-distribution

How is the result in `R`?

How is the result in `R`?