In statistics, the Fisher–Tippett–Gnedenko theorem (also the Fisher–Tippett theorem or the extreme value theorem) is a general result in extreme value theory regarding asymptotic distribution of extreme order statistics. The maximum of a sample of iid random variables after proper renormalization can only converge in distribution to one of only 3 possible distribution families: the Gumbel distribution, the Fréchet distribution, or the Weibull distribution. Credit for the extreme value theorem and its convergence details are given to Fréchet (1927),[1] Fisher and Tippett (1928),[2] Mises (1936),[3][4] and Gnedenko (1943).[5]
The role of the extremal types theorem for maxima is similar to that of central limit theorem for averages, except that the central limit theorem applies to the average of a sample from any distribution with finite variance, while the Fisher–Tippet–Gnedenko theorem only states that if the distribution of a normalized maximum converges, then the limit has to be one of a particular class of distributions. It does not state that the distribution of the normalized maximum does converge.
Statement
Let
be an n-sized sample of independent and identically-distributed random variables, each of whose cumulative distribution function is
Suppose that there exist two sequences of real numbers
and
such that the following limits converge to a non-degenerate distribution function:
![{\displaystyle \lim _{n\to \infty }{\boldsymbol {\mathcal {P}}}\left\{{\frac {\ \max\{X_{1},\dots ,X_{n}\}-b_{n}\ }{a_{n}}}\leq x\ \right\}=G(x)\ ,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/df2b62040fe32d8914a267007a34fe54d4bea7cf)
or equivalently:
![{\displaystyle \lim _{n\to \infty }{\Bigl (}\ F\left(\ a_{n}\ x+b_{n}\ \right){\Bigr )}^{n}=G(x)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6ccfc0b7aa40b44268f148c1a8af88bd3497a924)
In such circumstances, the limiting distribution
belongs to either the Gumbel, the Fréchet, or the Weibull distribution family.[6]
In other words, if the limit above converges, then up to a linear change of coordinates
will assume either the form:[7]
for ![{\displaystyle \quad \gamma \neq 0\ ,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/94ba5c4be3954f15bd590d64404bd19c5164251e)
with the non-zero parameter
also satisfying
for every
value supported by
(for all values
for which
). Otherwise it has the form:
for ![{\displaystyle \quad \gamma =0~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6c47df1db7348e77918877ff5e54587429d491a9)
This is the cumulative distribution function of the generalized extreme value distribution (GEV) with extreme value index
The GEV distribution groups the Gumbel, Fréchet, and Weibull distributions into a single composite form.
Conditions of convergence
The Fisher–Tippett–Gnedenko theorem is a statement about the convergence of the limiting distribution
above. The study of conditions for convergence of
to particular cases of the generalized extreme value distribution began with Mises (1936)[3][5][4] and was further developed by Gnedenko (1943).[5]
- Let
be the distribution function of
and
be some i.i.d. sample thereof.
- Also let
be the population maximum: ![{\displaystyle \ x_{\mathsf {max}}\equiv \sup \ \{\ x\ \mid \ F(x)<1\ \}~.\ }](https://wikimedia.org/api/rest_v1/media/math/render/svg/55c492e5e7c3e87139672018d6b8bdb2f1f3277a)
The limiting distribution of the normalized sample maximum, given by
above, will then be:[7]
- Fréchet distribution
![{\displaystyle \ \left(\ \gamma >0\ \right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/02e4883ae1ba7ecaa6d3f881d092a19702195ce7)
- For strictly positive
the limiting distribution converges if and only if ![{\displaystyle \ x_{\mathsf {max}}=\infty \ }](https://wikimedia.org/api/rest_v1/media/math/render/svg/6285e143ca64ab6dacd86759a33c8aaaf4bf1288)
- and
for all ![{\displaystyle \ u>0~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/1414d486ae7574a6cb448298fc5cc9e6c94ebab8)
- In this case, possible sequences that will satisfy the theorem conditions are
![{\displaystyle b_{n}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/ecc141d40ca4a13ec7beafd6d264d5a435792a58)
- and
![{\displaystyle \ a_{n}={F^{-1}}\!\!\left(1-{\tfrac {1}{\ n\ }}\right)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/cb1adacc893258c4978f1d9a1c120681b1d3875e)
- Strictly positive
corresponds to what is called a heavy tailed distribution.
- Gumbel distribution
![{\displaystyle \ \left(\ \gamma =0\ \right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/dce1191311f5592cd85fc781001ba07dd1796fb5)
- For trivial
and with
either finite or infinite, the limiting distribution converges if and only if
for all ![{\displaystyle \ u>0\ }](https://wikimedia.org/api/rest_v1/media/math/render/svg/9a8235215da146e1a590e3e6b335242663a6a38e)
- with
![{\displaystyle \ {\tilde {g}}(t)\equiv {\frac {\ \int _{t}^{x_{\mathsf {max}}}{\Bigl (}\ 1-F(s)\ {\Bigr )}\ \mathrm {d} \ s\ }{1-F(t)}}~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3b475502f380a29626a97eb58cd151c7b310e81c)
- Possible sequences here are
![{\displaystyle \ b_{n}={F^{-1}}\!\!\left(\ 1-{\tfrac {1}{\ n\ }}\ \right)\ }](https://wikimedia.org/api/rest_v1/media/math/render/svg/1c47032fe8ecc5be9684084556a7110a5b746682)
- and
![{\displaystyle \ a_{n}={\tilde {g}}{\Bigl (}\;{F^{-1}}\!\!\left(\ 1-{\tfrac {1}{\ n\ }}\ \right)\;{\Bigr )}~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/23f7dfa4b210358d868bc6cd0db986f729c7da84)
- Weibull distribution
![{\displaystyle \ \left(\ \gamma <0\ \right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/94a6f75d6d57364dbba56e6c75f832ca867ab840)
- For strictly negative
the limiting distribution converges if and only if
(is finite)
- and
for all ![{\displaystyle \ u>0~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/1414d486ae7574a6cb448298fc5cc9e6c94ebab8)
- Note that for this case the exponential term
is strictly positive, since
is strictly negative. - Possible sequences here are
![{\displaystyle \ b_{n}=x_{\mathsf {max}}\ }](https://wikimedia.org/api/rest_v1/media/math/render/svg/4822a82d2dd27c1b547a5a93c7ee9549b6bab359)
- and
![{\displaystyle \ a_{n}=x_{\mathsf {max}}-{F^{-1}}\!\!\left(\ 1-{\frac {1}{\ n\ }}\ \right)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/cbdf16519b9c329fd1626381f883a87d70966caa)
Note that the second formula (the Gumbel distribution) is the limit of the first (the Fréchet distribution) as
goes to zero.
Examples
Fréchet distribution
The Cauchy distribution's density function is:
![{\displaystyle f(x)={\frac {1}{\ \pi ^{2}+x^{2}\ }}\ ,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7c526c3f021d25c3a1d517c0c87370a7de868644)
and its cumulative distribution function is:
![{\displaystyle F(x)={\frac {\ 1\ }{2}}+{\frac {1}{\ \pi \ }}\arctan \left({\frac {x}{\ \pi \ }}\right)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/ef6bcadcaf1d9048a76703ca75cf47fd797fce5d)
A little bit of calculus show that the right tail's cumulative distribution
is asymptotic to
or
![{\displaystyle \ln F(x)\rightarrow {\frac {-1~}{\ x\ }}\quad {\mathsf {~as~}}\quad x\rightarrow \infty \ ,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e6012803a7c7d5e03289691845fde89fbd0eeedc)
so we have
![{\displaystyle \ln \left(\ F(x)^{n}\ \right)=n\ \ln F(x)\sim -{\frac {-n~}{\ x\ }}~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/024046f124a501646d86b2dd1f88b1b49d7ca856)
Thus we have
![{\displaystyle F(x)^{n}\approx \exp \left({\frac {-n~}{\ x\ }}\right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/69233bf38db0161d8900596e38091afa0e839c7c)
and letting
(and skipping some explanation)
![{\displaystyle \lim _{n\to \infty }{\Bigl (}\ F(n\ u+n)^{n}\ {\Bigr )}=\exp \left({\tfrac {-1~}{\ 1+u\ }}\right)=G_{1}(u)\ }](https://wikimedia.org/api/rest_v1/media/math/render/svg/b8184639c9709a66f6ce29298b0fbad54253883b)
for any
Gumbel distribution
Let us take the normal distribution with cumulative distribution function
![{\displaystyle F(x)={\frac {1}{2}}\operatorname {erfc} \left({\frac {-x~}{\ {\sqrt {2\ }}\ }}\right)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d7501a94e9dfa2aff1b25d5e935e1aa855fe532f)
We have
![{\displaystyle \ln F(x)\rightarrow -{\frac {\ \exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\quad {\mathsf {~as~}}\quad x\rightarrow \infty }](https://wikimedia.org/api/rest_v1/media/math/render/svg/185b7d54ef9367fd8f08180ea8391e662a199f45)
and thus
![{\displaystyle \ln \left(\ F(x)^{n}\ \right)=n\ln F(x)\rightarrow -{\frac {\ n\exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\quad {\mathsf {~as~}}\quad x\rightarrow \infty ~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4bac900406d1349f924faae82f291217853332c4)
Hence we have
![{\displaystyle F(x)^{n}\approx \exp \left(-\ {\frac {\ n\ \exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{\ {\sqrt {2\pi \ }}\ x\ }}\right)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/8fd7998a4b2bb5cbd377ad65f1ada44b5bb68025)
If we define
as the value that exactly satisfies
![{\displaystyle {\frac {\ n\exp \left(-\ {\tfrac {1}{2}}c_{n}^{2}\right)\ }{\ {\sqrt {2\pi \ }}\ c_{n}\ }}=1\ ,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c1946eecd155f0078b133c6331086d7bebc69de9)
then around
![{\displaystyle {\frac {\ n\ \exp \left(-\ {\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\approx \exp \left(\ c_{n}\ (c_{n}-x)\ \right)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0c87e9b2b6bcff6c70a4776ccc47472cf7c51ac1)
As
increases, this becomes a good approximation for a wider and wider range of
so letting
we find that
![{\displaystyle \lim _{n\to \infty }{\biggl (}\ F\left({\tfrac {u}{~c_{n}\ }}+c_{n}\right)^{n}\ {\biggr )}=\exp \!{\Bigl (}-\exp(-u){\Bigr )}=G_{0}(u)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/a037c5b7b89aa97a04b1bad8da9ec91c2e82daae)
Equivalently,
![{\displaystyle \lim _{n\to \infty }{\boldsymbol {\mathcal {P}}}\ {\Biggl (}{\frac {\ \max\{X_{1},\ \ldots ,\ X_{n}\}-c_{n}\ }{\left({\frac {u}{~c_{n}\ }}\right)}}\leq u{\Biggr )}=\exp \!{\Bigl (}-\exp(-u){\Bigr )}=G_{0}(u)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d6a83e1794330439c0aada0657abe3477ff0d516)
With this result, we see retrospectively that we need
and then
![{\displaystyle c_{n}\approx {\sqrt {2\ln n\ }}\ ,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f1085942d743ecf8afd4a07495447a45404213b5)
so the maximum is expected to climb toward infinity ever more slowly.
Weibull distribution
We may take the simplest example, a uniform distribution between 0 and 1, with cumulative distribution function
for any x value from 0 to 1 .
For values of
we have
![{\displaystyle \ln {\Bigl (}\ F(x)^{n}\ {\Bigr )}=n\ \ln F(x)\ \rightarrow \ n\ (\ 1-x\ )~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/93c5d55eabb4b8233db25094ad553bb0aa19c1ac)
So for
we have
![{\displaystyle \ F(x)^{n}\approx \exp(\ n-n\ x\ )~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/234894afde7e7ca6e5537636ee7b040d522f5d9c)
Let
and get
![{\displaystyle \lim _{n\to \infty }{\Bigl (}\ F\!\left({\tfrac {\ u\ }{n}}+1-{\tfrac {\ 1\ }{n}}\right)\ {\Bigr )}^{n}=\exp \!{\bigl (}\ -(1-u)\ {\bigr )}=G_{-1}(u)~.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3c153c0d4388377adacecc4fbfc918044c2c9864)
Close examination of that limit shows that the expected maximum approaches 1 in inverse proportion to n .
See also
References
- ^ Fréchet, M. (1927). "Sur la loi de probabilité de l'écart maximum". Annales de la Société Polonaise de Mathématique. 6 (1): 93–116.
- ^ Fisher, R.A.; Tippett, L.H.C. (1928). "Limiting forms of the frequency distribution of the largest and smallest member of a sample". Proc. Camb. Phil. Soc. 24 (2): 180–190. Bibcode:1928PCPS...24..180F. doi:10.1017/s0305004100015681. S2CID 123125823.
- ^ a b von Mises, R. (1936). "La distribution de la plus grande de n valeurs" [The distribution of the largest of n values]. Rev. Math. Union Interbalcanique. 1 (in French): 141–160.
- ^ a b Falk, Michael; Marohn, Frank (1993). "von Mises conditions revisited". The Annals of Probability: 1310–1328.
- ^ a b c Gnedenko, B.V. (1943). "Sur la distribution limite du terme maximum d'une serie aleatoire". Annals of Mathematics. 44 (3): 423–453. doi:10.2307/1968974. JSTOR 1968974.
- ^ Mood, A.M. (1950). "5. Order Statistics". Introduction to the theory of statistics. New York, NY: McGraw-Hill. pp. 251–270.
- ^ a b Haan, Laurens; Ferreira, Ana (2007). Extreme Value Theory: An introduction. Springer.
Further reading
- Lee, Seyoon; Kim, Joseph H.T. (8 March 2018). "Exponentiated generalized Pareto distribution". Communications in Statistics – Theory and Methods. 48 (8) (online ed.): 2014–2038. arXiv:1708.01686. doi:10.1080/03610926.2018.1441418. ISSN 1532-415X – via tandfonline.com.