When working with random variables, you might be confronted with the need to compute the distributions or statistics (e.g. mean) related to the $$\min$$ and $$\max$$ of those random variables. This could come up in the context of queuing theory, failure analysis, extreme value theory, etc. While you can find a large body of literature related to the order statistic, here we will focus on the two special cases of $$\min$$ and $$\max$$. Let's first derive the distribution functions of the $$\min$$ and $$\max$$ in general terms and then illustrate with a couple of examples (uniform and exponential distributions). 

Problem setup

Let $$X_1, X_2,\; ... \;,X_N$$ denote $$N$$ independent and identically distributed random variables. Let $$f_X(x)$$ denote the probability density function (PDF) and $$F_X(x)$$ denote the cumulative density function (CDF) associated with these random variables. We will focus on continuous random variables here, though the ideas extend to the discrete case. Let $$Y = \min \{ X_1, X_2,\; ... \;,X_N \}$$ denote the minimum of all $$N$$ random variables and let $$Z = \max \{ X_1, X_2,\; ... \;,X_N \}$$ denote the maximum of all $$N$$ random variables.

Distribution of the max

To compute the CDF of $$Z$$, we need to compute the probability $$P(Z \leq z)$$. The key is to note that when $$Z \leq z$$, all $$N$$ random variables are less than or equal to $$z$$, because $$Z$$ is the largest of all these random variables. Thus,

$$F_Z(z) = P(Z \leq z)$$

$$= P(X_1 \leq z, X_2 \leq z, \; ... \; X_N \leq z)$$

$$ = P(X_1 \leq z)P(X_2 \leq z)...P(X_N \leq z)$$ (since the random variables are independent)

$$ = P(X_1 \leq z)^N$$ (since the random variables are identically distributed)

$$ = F_X(z)^N$$.

We can now compute the PDF of $$Z$$, $$f_Z(z)$$, by computing the derivative of the CDF (by definition). 

$$f_Z(z) = \frac{d}{dz} F_Z(z) $$

$$={d \over dz}F_X(z)^N$$

$$=NF_X(z)^{N-1}{d \over dz}F_X(z)$$

$$=NF_X(z)^{N-1}f_X(z)$$.

Distribution of the min

To compute the CDF of $$Y$$, we need to compute the probability $$P(Y \leq y)$$, which can be rewritten as $$1-P(Y>y)$$. This time, the key is to note that when $$Y > y$$, all $$N$$ random variables are greater than $$y$$, because $$Y$$ is the smallest of all these random variables. Thus,

$$F_Y(y) = P(Y \leq y)$$

$$=1-P(Y>y)$$

$$= 1-P(X_1 > y, X_2 > y, \; ... \; X_N > y)$$

$$ = 1-P(X_1 > y)P(X_2 > y)...P(X_N > y)$$ (since the random variables are independent)

$$ = 1-P(X_1 > y)^N$$ (since the random variables are identically distributed)

$$ = 1-\left( 1-F_X(y) \right)^N$$.

We can now compute the PDF of $$Y$$, $$f_Y(y)$$, by computing the derivative of the CDF (by definition). 

$$f_Y(y) = {d \over dy} F_Y(y)$$

$$={d \over dy}\left[ 1-\left( 1-F_X(y) \right)^N \right]$$

$$=N\left( 1-F_X(y) \right)^{N-1}{d \over dy}F_X(y)$$

$$=N\left( 1-F_X(y) \right)^{N-1}f_X(y)$$.

Example 1

Let's consider the case where the $$X_i$$s are uniformly distributed over $$[0,1]$$, denoted $$X_i \sim U[0,1]$$. The PDF is given by $$f_X(x) = 1, \; x \in [0,1]$$ and the CDF is given by $$F_X(x) = x, \; x \in [0,1]$$. Using the results derived above, the PDF of the $$\max$$ is given by $$f_Z(z) = Nz^{N-1}, \; z \in [0,1]$$ and the PDF of the $$\min$$ is given by $$f_Y(y) = N(1-y)^{N-1}, \; y \in [0,1]$$. We can now easily compute the mean (or expected value) of the $$\max$$ and $$\min$$ as follows:

$$\mathbb{E}[Z] = \int_0^1 z f_Z(z) dz = \int_0^1 Nz^N dz = {N \over N+1}$$ (since $$\int z^n dz = {z^{n+1} \over n+1}$$).

$$\mathbb{E}[Y] = \int_0^1 yf_Y(y) dy = \int_0^1 Ny(1-y)^{N-1} dy$$

By setting $$u=1-y$$ (and $$du = -dy$$), we can simplify the above integral as

$$\mathbb{E}[Y] = \int_0^1 N(1-u)u^{N-1}du = \int_0^1 Nu^{N-1}du - \int_0^1 Nu^N du = 1 - {N \over N+1} = {1 \over N+1}$$.

It is intuitively satisfying to note that the expected value of the $$\max$$ approaches 1 and the expected value of the $$\min$$ approaches 0 as $$N \rightarrow \infty$$. For $$N=1$$, the mean of both the $$\min$$ and $$\max$$ are $$1/2$$, as expected. Finally, you might be able to guess from the above analysis that the mean of the $$k^{th}$$ smallest of these $$N$$ random variables is $${k \over N+1}$$.

Example 2

In our second example, let's consider the case where the $$X_i$$ are exponentially distributed with mean 1, denoted $$X_i \sim exp(1)$$. The PDF is given by $$f_X(x) = e^{-x}, \; x \geq 0$$ and the CDF is given by $$F_X(x) = 1 - e^{-x}, \; x \geq 0$$. Using the results derived above, the PDF of the $$\max$$ is given by $$f_Z(z) = Ne^{-z}(1-e^{-z})^{N-1}$$ and the PDF of the $$\min$$ is given by $$f_Y(y) = Ne^{-(N-1)y}e^{-y} = Ne^{-Ny}$$. It is interesting to note that the PDF of $$Y$$ takes the form of the exponential distribution with parameter $$N$$. This implies that the $$\min$$ of $$N$$ independent and identically distributed exponential random variables with mean 1 is also an exponential random variable with mean $$1/N$$. Cool! The $$\max$$ clearly doesn't follow an exponential distribution, but let's try to compute it's mean.

 $$\mathbb{E}[Z] = \int_0^{\infty} Ne^{-z}(1-e^{-z})^{N-1} dz$$

$$ = \int_0^{\infty} zNe^{-z} \sum_{k=0}^{N-1}{N-1 \choose k} e^{-kz}(-1)^k dz$$ (binomial expansion of $$(1-x)^n$$)

$$ = N\sum_{k=0}^{N-1} (-1)^k {N-1 \choose k} \int_0^{\infty} z e^{-(k+1)z} dz $$ (switching the order of integration and summation, under some technical assumptions)

$$ = N\sum_{k=0}^{N-1}(-1)^k {N-1 \choose k}{1 \over (k+1)^2} $$ (using the result $$\int_0^\infty x e^{-\alpha x} dx = {1 \over \alpha^2}$$, which we will prove at the end of this post)

$$ = N\sum_{k=0}^{N-1}(-1)^k {(N-1)! \over k!(N-1-k)!}{1 \over (k+1)^2} $$ (using the definition of $${n \choose k}$$)

$$=\sum_{k=0}^{N-1}(-1)^k{N! \over (k+1)!(N-k-1)!}{1 \over k+1}$$ (using $$n! = n(n-1)!$$)

$$=\sum_{k=1}^N(-1)^{k-1}{N \choose k}{1 \over k}$$.

The result is quite compact though not very elegant. While I don't know of a way to manipulate the above expression directly to get to it, there are other ways to show that $$\mathbb{E}[Z] = \sum_{k=1}^N {1 \over k}$$, i.e. the $$N^{th}$$ harmonic number. A much more elegant result! We will attempt the proof in a separate blog post.

Wrapping up

To wrap up, let me show that $$\int_0^\infty x e^{-\alpha x} dx = {1 \over \alpha^2}$$, as promised above. Actually it can be shown easily using the integration by parts rule, which says $$\int u(x)dv(x) = u(x)v(x) - \int v(x)du(x)$$. In our case, $$u(x) = x$$ and $$v(x) = -{1 \over \alpha}e^{-\alpha x}$$. Substituting, we get

$$\int x e^{-\alpha x} dx = -{x \over \alpha}e^{-\alpha x} + {1 \over \alpha} \int e^{-\alpha x} dx = -{x \over \alpha}e^{-\alpha x} - {1 \over \alpha^2} e^{-\alpha x}$$.

Now, applying the limits of integration (0 to $$\infty$$) and using $$\lim_{x \rightarrow \infty} xe^{-\alpha x} = 0 \; \forall \alpha>0$$, we get the desired result $${1 \over \alpha^2}$$.