Probability and Statistics

Derivation. Derives Proposition 11.2. Every claim is a statement about a series of non-negative terms indexed by a countable set, and such a series may be split and regrouped at will. (i) Splitting the sum Equation (11.1) over \(A\) and its complement gives the first identity; splitting the sum over \(B\) over \(A\) and \(B\setminus A\) gives \(\Pr(B)=\Pr(A)+\Pr(B\setminus A)\ge\Pr(A)\). (ii) The empty sum is \(0\) and the sum over \(\Omega\) is \(1\) by Equation (11.1); the bounds then follow from monotonicity, \(\Pr(\varnothing)\le\Pr(A)\le\Pr(\Omega)\). (iii) Split \(A\cup B\) into \(A\setminus B\), \(B\setminus A\) and \(A\cap B\): the left side is the sum of the three, and the right side counts \(A\cap B\) twice and subtracts it once. (iv) Each outcome of the union lies in exactly one \(A_{i}\), so grouping the terms of the sum over the union by the index \(i\) reproduces the right-hand side.

Derivation. Derives Proposition 11.5. (i) Group the outcomes according to the value of \(X\): the terms of \(\sum_{\omega}g(X(\omega))\Pr(\set{\omega})\) with \(X(\omega)=x\) contribute \(g(x)\sum_{\omega:X(\omega)=x}\Pr(\set{\omega})=g(x)p_{X}(x)\), and the regrouping is legitimate because the series converges absolutely by hypothesis. (ii) Termwise, \(\left(aX+bY\right)(\omega)\Pr(\set{\omega}) =aX(\omega)\Pr(\set{\omega})+bY(\omega)\Pr(\set{\omega})\); both series on the right converge absolutely, so their sum may be split, and \(\abs{aX+bY}\le\abs{a}\abs{X}+\abs{b}\abs{Y}\) gives the integrability. For a constant the sum is \(c\) times Equation (11.1). (iii) Termwise again, since each weight is non-negative; and \(\pm X\le\abs{X}\) gives \(\pm\avg{X}\le\avg{\abs{X}}\).

Derivation. Derives Proposition 11.7. Expand the square and use linearity twice: \(\avg{X^{2}-2\mu X+\mu^{2}}=\avg{X^{2}}-2\mu\avg{X}+\mu^{2} =\avg{X^{2}}-\mu^{2}\), since \(\avg{X}=\mu\). For the second claim, \(\avg{aX+b}=a\mu+b\), so \(\left(aX+b\right)-\avg{aX+b}=a\left(X-\mu\right)\) and squaring produces the factor \(a^{2}\). For the third, \(\operatorname{var}X=\sum_{\omega}\left(X(\omega)-\mu\right)^{2} \Pr(\set{\omega})\) is a series of non-negative terms, so it vanishes exactly when every term does, that is when \(X(\omega)=\mu\) at every outcome of non-zero probability.

Derivation. Derives Proposition 11.10. For Equation (11.8) write \(X-c=\left(X-\mu\right)+\left(\mu-c\right)\) and expand; the cross term is \(2(\mu-c)\avg{X-\mu}=0\).

For Equation (11.7) take first \(c>m\) and put \(d=c-m>0\). Then

Indeed, for \(x\le m<c\) both sides equal \(d\), because \(\abs{x-c}-\abs{x-m}=(c-x)-(m-x)=d\); and for \(x>m\) the right-hand side is \(-d\), while the left-hand side is at least \(-\abs{c-m}=-d\) by the triangle inequality. Put \(x=X\) in Equation (11.9) and take expectations, using monotonicity and linearity (Proposition 11.5):

since \(\Pr(X>m)=1-\Pr(X\le m)\le\tfrac12\) by Equation (11.6) at \(\alpha=\tfrac12\). For \(c<m\) the same argument with \(d=m-c\) and the indicator of \(\set{x<m}\) applies, using \(\Pr(X<m)=1-\Pr(X\ge m)\le\tfrac12\).

Derivation. Derives Proposition 11.12. The first two claims are Equation (11.4) and the vanishing of \(c-\avg{c}\). For bilinearity, the centred variable of \(aX+bY\) is \(a\left(X-\avg{X}\right)+b\left(Y-\avg{Y}\right)\) by linearity of the expectation, and multiplying by \(Z-\avg{Z}\) and taking expectations splits the result by linearity again; symmetry then gives linearity in the second slot. Expanding Equation (11.10) gives \(\avg{XY}-\avg{X}\avg{Y}-\avg{Y}\avg{X}+\avg{X}\avg{Y} =\avg{XY}-\avg{X}\avg{Y}\). Finally Equation (11.12) is bilinearity applied to \(\operatorname{cov}\!\left(\sum_{i}a_{i}X_{i}, \sum_{j}a_{j}X_{j}\right)\), the diagonal terms giving the variances and the off-diagonal ones pairing up by symmetry.

Derivation. Derives Proposition 11.14. Grouping the outcomes by the value of the pair \((X,Y)\), exactly as in Proposition 11.5(i), gives \(\avg{XY}=\sum_{x,y}xy\Pr(X=x,Y=y)\), and the double series converges absolutely because \(\sum_{x,y}\abs{x}\abs{y}p_{X}(x)p_{Y}(y) =\avg{\abs{X}}\avg{\abs{Y}}<\infty\). Absolute convergence permits summing in either order and splitting the product of the two series (Proposition 7.49), so Equation (11.13) gives \(\avg{XY}=\left(\sum_{x}x\,p_{X}(x)\right) \left(\sum_{y}y\,p_{Y}(y)\right)=\avg{X}\avg{Y}\), and Proposition 11.12 makes the covariance vanish. For the converse, let \(X\) take the values \(-1,0,1\) with probability \(\tfrac13\) each and put \(Y=X^{2}\). Then \(\avg{X}=0\) and \(\avg{XY}=\avg{X^{3}}=0\), so the covariance vanishes; but \(\Pr(X=0,Y=0)=\tfrac13\) while \(p_{X}(0)\,p_{Y}(0)=\tfrac13\cdot\tfrac13\), so Equation (11.13) fails and \(Y\) is in fact a function of \(X\).

Derivation. Derives Corollary 11.15. Divide Equation (11.50) by the positive number \(\operatorname{var}X\cdot\operatorname{var}Y\) and take square roots. Equality holds exactly under the condition recorded in Lemma 11.63: there are constants \((a_{0},c_{0})\neq(0,0)\) with \(a_{0}\left(X-\avg{X}\right)+c_{0}\left(Y-\avg{Y}\right)=0\) with probability one. Neither coefficient can vanish here, because \(a_{0}=0\) would force \(\operatorname{var}Y=0\) and \(c_{0}=0\) would force \(\operatorname{var}X=0\). Hence \(Y-\avg{Y}=k\left(X-\avg{X}\right)\) with \(k=-a_{0}/c_{0}\neq0\), which is the stated affine relation with \(b=\avg{Y}-k\avg{X}\). For such a \(Y\), Proposition 11.12 gives \(\operatorname{cov}(X,Y)=k\operatorname{var}X\) and \(\operatorname{var}Y=k^{2}\operatorname{var}X\), so \(\rho=k/\abs{k}=\sgn k\).

Derivation. Derives Proposition 11.17. The first identity is linearity. The second is Equation (11.12) with \(a_{i}=1/n\): the covariances vanish by Proposition 11.14, leaving \(n\cdot n^{-2}\sigma^{2}\). For the third, write \(X_{i}-\bar{X}_{n}=\left(X_{i}-\mu\right)-\left(\bar{X}_{n}-\mu\right)\) and sum the squares; since \(\sum_{i}\left(X_{i}-\mu\right) =n\left(\bar{X}_{n}-\mu\right)\), the cross term is \(-2n\left(\bar{X}_{n}-\mu\right)^{2}\) and

Taking expectations gives \(n\sigma^{2}-n\cdot\sigma^{2}/n =(n-1)\sigma^{2}\), and dividing by \(n-1\) gives \(\avg{s_{n}^{2}}=\sigma^{2}\). Finally \(0\le\operatorname{var}s_{n}=\avg{s_{n}^{2}}-\avg{s_{n}}^{2} =\sigma^{2}-\avg{s_{n}}^{2}\) by Equation (11.5), so \(\avg{s_{n}}\le\sigma\), with equality exactly when \(\operatorname{var}s_{n}\) vanishes.

Derivation. Derives Proposition 11.22. Monotonicity and the difference formula follow from additivity of the integral over subintervals together with \(f\ge0\); the two limits are Equation (11.19). A Riemann integrable function is bounded on each bounded interval, say by \(M\) near \(x\), so \(\abs{F_{X}(x+h)-F_{X}(x)}\le M\abs{h}\) and \(F_{X}\) is continuous — indeed locally Lipschitz. Hence \(\Pr(X=x)\le F_{X}(x+\varepsilon)-F_{X}(x-\varepsilon)\rightarrow0\). The last claim is the first fundamental theorem of calculus (Theorem 7.42), which needs \(f\) continuous only at the point in question.

Derivation. Derives Proposition 11.24. Linearity and monotonicity of the integral are the elementary properties of the Darboux construction recorded with Definition 7.39, and applying monotonicity to \(\pm g f\le\abs{g}f\) gives the third property. Reading the proofs of the five results named, each uses the expectation only through those three properties together with pointwise algebraic identities and the pointwise inequality Equation (11.9); none of them rearranges a series. The same holds of Lemma 11.63, whose proof needs only that \(\avg{\left(s\tilde{V}+\tilde{W}\right)^{2}}\ge0\), and which is in any case stated for arbitrary random variables of finite variance. One dependence is not of that kind and has to be replaced rather than reread: Proposition 11.17 also uses the vanishing of the covariance of two independent variables, which here is Proposition 11.26 below and not Proposition 11.14.

Derivation. Derives Proposition 11.26. In the iterated integral \(\avg{XY}=\int\!\left(\int xy\,f_{X}(x)f_{Y}(y)\,\dd y\right)\dd x\) the inner integrand is \(x f_{X}(x)\) times a function of \(y\) alone, so the inner integral equals \(x f_{X}(x)\int y f_{Y}(y)\,\dd y\) — a constant multiple of \(x f_{X}(x)\), the constant being \(\avg{Y}\), which is finite by hypothesis. Integrating in \(x\) gives \(\avg{X}\avg{Y}\). No interchange of integrations is used: the factorisation is of the integrand, not of the order of integration. Proposition 11.12, which transfers by Proposition 11.24, then makes the covariance vanish.

Derivation. Derives Proposition 11.27. \(F\) is continuous by Proposition 11.22 and strictly increasing by hypothesis, so it has a continuous strictly increasing inverse on \((0,1)\) and \(\set{F(X)\le\alpha}=\set{X\le F^{-1}(\alpha)}\). Hence \(\Pr(U\le\alpha)=F\!\left(F^{-1}(\alpha)\right)=\alpha\). Conversely \(\Pr\!\left(F^{-1}(U)\le x\right)=\Pr\!\left(U\le F(x)\right)=F(x)\).

Derivation. Derives Proposition 11.29. The substitution \(u=(x-\mu)/\sigma\) (Equation (7.27)) reduces every statement to the standard case, with \(\mu_{n}=\sigma^{n}\avg{U^{n}}\) for \(U\) standard. For the normalisation put \(I=\int_{-\infty}^{\infty}\ee^{-u^{2}/2}\,\dd u\) and evaluate \(I^{2}\) as a double integral in polar coordinates, whose Jacobian is \(r\):

so \(I=\sqrt{2\pi}\) and \(\varphi\) integrates to \(1\). This is the one step of the derivation that uses the change-of-variables formula for a double integral.

The odd moments vanish because \(u^{2n+1}\varphi(u)\) is odd and absolutely integrable. For the even ones use \(\varphi'(u)=-u\varphi(u)\) and integrate by parts (Equation (7.28)):

the boundary term vanishing because \(u^{2n-1}\varphi(u)\rightarrow0\) at both ends. With \(\avg{U^{0}}=1\) this gives \(\avg{U^{2n}}=(2n-1)!!\), hence \(\avg{U^{2}}=1\) and \(\avg{U^{4}}=3\). So the mean is \(\mu\), the variance is \(\sigma^{2}\), \(\gamma_{1}=0\) and \(\gamma_{2}=3-3=0\), which is the convention fixed in Definition 11.6.

Derivation. Derives Proposition 11.36. (i) Write \(B=A\cup(B\setminus A)\), a disjoint union of measurable sets; Equation (11.26) with all later terms empty gives \(\mu(B)=\mu(A)+\mu(B\setminus A)\ge\mu(A)\), the last step because a measure is non-negative.

(ii) Disjointify: put \(B_{1}=A_{1}\) and \(B_{n}=A_{n}\setminus(A_{1}\cup\dots\cup A_{n-1})\), which lie in \(\mathcal{F}\) by Definition 11.33, are pairwise disjoint, have the same union as the \(A_{n}\), and satisfy \(B_{n}\subseteq A_{n}\). Then \(\mu\left(\bigcup_{n}A_{n}\right)=\sum_{n}\mu(B_{n}) \le\sum_{n}\mu(A_{n})\) by Equation (11.26) and (i).

(iii) If every \(\mu(A_{n})=0\) then the right-hand side of (ii) is a series of zeros, so the union has measure at most zero, hence exactly zero.

(iv) Apply (iii) to the union of the countably many exceptional sets: outside it, every one of the statements holds.

Derivation. Derives Proposition 11.43. (i) \(\varphi_X(0)=\avg{1}=1\), and \(\abs{\varphi_X(t)} = \abs{\avg{\ee^{\ii tX}}} \le \avg{\abs{\ee^{\ii tX}}} = 1\). (ii) \(\avg{\ee^{\ii t(aX+b)}} = \ee^{\ii bt}\avg{\ee^{\ii (ta)X}}\). (iii) \(\ee^{\ii t(X+Y)} = \ee^{\ii tX}\ee^{\ii tY}\), and if \(X\) and \(Y\) are independent then \(\avg{f(X)g(Y)} = \avg{f(X)}\avg{g(Y)}\) for all bounded measurable \(f,g\) — which is what independence means, and is strictly stronger than being uncorrelated: the latter is the case \(f=g=\id\) alone and would not suffice here, since \(\ee^{\ii tX}\) is a nonlinear function of \(X\). Both factors are bounded, so both expectations exist.

Derivation. Derives Lemma 11.44. Writing the remainder of \(\ee^{\ii x}\) in integral form by integrating by parts \(n\) times gives, for real \(x\),

Bounding the integrand of that remainder gives the first estimate; the second follows instead from the triangle inequality applied to the order-\((n-1)\) remainder together with the last term of the series. The minimum matters because only the second branch survives when no moment beyond the \(n\)th exists. Taking \(n=2\) and \(x=tX\),

Take expectations, and use \(\abs{\avg{Z}}\le\avg{\abs{Z}}\). Since \(\avg{X}=0\) the left side becomes \(\abs{\varphi_{X}(t) - 1 + \tfrac12 t^{2}\sigma^{2}}\), so

The right-hand side tends to \(0\) as \(t\rightarrow0\). The usual reason given is dominated convergence, which this treatise cannot yet invoke: Real Analysis develops the Riemann integral only, and the Lebesgue theory the dominated-convergence theorem belongs to is not built anywhere in the book. An elementary truncation does the same work. Fix \(K>0\) and split the expectation at \(\abs{X}=K\): where \(\abs{X}\le K\) the first branch of the minimum bounds the integrand by \(\tfrac{1}{6}\abs{t}K^{3}\), and where \(\abs{X}>K\) the second branch bounds it by \(X^{2}\indic{\abs{X}>K}\), so

From Equation (11.31) (take \(n=2\) and \(x=tX\), then split the expectation at \(\abs{X}=K\) and use one branch of the minimum on each piece). Given \(\varepsilon>0\), choose \(K\) so large that the second term is below \(\varepsilon/2\) — possible precisely because \(\avg{X^{2}}\) is finite, a finite integral being one whose tail contribution vanishes [Billingsley:1995] — and then take \(\abs{t}<3\varepsilon/K^{3}\), which puts the first term below \(\varepsilon/2\) as well. That is exactly Equation (11.30).

Derivation. Derives Theorem 11.46. Put \(Y_{i}=(X_{i}-\mu)/\sigma\), so the \(Y_{i}\) are independent and identically distributed with \(\avg{Y_{i}}=0\) and \(\avg{Y_{i}^{2}}=1\), and \(Z_{n}=n^{-1/2}\sum_{i}Y_{i}\). Write \(\varphi\) for the common characteristic function of the \(Y_{i}\).

By Proposition 11.43(ii) and (iii), the latter extended to \(n\) factors by induction,

Fix \(t\). By Lemma 11.44 applied at argument \(t/\sqrt{n}\),

From Equation (11.30) (the second-order expansion evaluated at argument \(t/\sqrt{n}\)). The remainder \(r_{n}(t)\) is defined by that equation; the convergence is the lemma rescaled. Writing \(\varphi(s) = 1 - s^{2}/2 + R(s)\) with \(R(s)/s^{2}\rightarrow0\), we have \(r_{n}(t) = R(t/\sqrt{n})\) and hence \(n\,r_{n}(t) = t^{2}\,R(s)/s^{2}\) at \(s=t/\sqrt{n}\), which tends to \(0\) for each fixed \(t\) because \(s\rightarrow0\). Now compare the two products using the elementary inequality

which follows by telescoping \(\prod a_{i} - \prod b_{i} = \sum_{i} \left(\prod_{j<i}a_{j}\right) (a_{i}-b_{i})\left(\prod_{j>i}b_{j}\right)\) and bounding each partial product by \(1\). Take \(a_{i}=\varphi(t/\sqrt{n})\), whose modulus is at most \(1\) by Proposition 11.43(i), and \(b_{i}=1-t^{2}/(2n)\), which lies in \([0,1]\) once \(n\ge t^{2}/2\). Then

Since \((1-t^{2}/(2n))^{n}\rightarrow\ee^{-t^{2}/2}\), it follows that

From Equations (11.34), (11.35) and (11.36) (compare the two products factor by factor and let \(n\) grow). The limit is the characteristic function of the standard Gaussian. That is quickest to see without contour shifting: writing \(g(t)=\int\ee^{\ii tu}\ee^{-u^{2}/2}\dd u/\sqrt{2\pi}\) and differentiating under the integral sign, an integration by parts gives \(g'(t) = -t\,g(t)\) with \(g(0)=1\), whence \(g(t)=\ee^{-t^{2}/2}\). It is continuous at \(t=0\). Theorem 11.45 therefore gives \(Z_{n}\rightarrow\mathcal{N}(0,1)\) in distribution.

Derivation. Derives Lemma 11.50. Since \(Y\ge0\), \(Y\ge Y\,\indic{Y\ge a}\ge a\,\indic{Y\ge a}\) pointwise; taking expectations gives \(\avg{Y}\ge a\Pr(Y\ge a)\). Apply this to \(Y=(X-\mu)^{2}\) with the level \(a^{2}\): the event \(\abs{X-\mu}\ge a\) is the event \(Y\ge a^{2}\), and \(\avg{Y}=\sigma^{2}\).

Derivation. Derives Corollary 11.51. \(\avg{\bar{X}_{n}}=\mu\) and, by independence, \(\operatorname{var}\bar{X}_{n}=\sigma^{2}/n\). Then Equation (11.41) gives \(\Pr(\abs{\bar{X}_{n}-\mu}\ge a)\le\sigma^{2}/(na^{2})\rightarrow0\).

Derivation of the sum and product clauses. Derives Lemma 11.52. Write \(F\) for the distribution function of \(A\) and \(F_{n}(y)=\Pr(A_{n}\le y)\), both in the general sense of Definition 11.21: no density is assumed here, and \(F\) may jump — as it does for the constant limits met below — which is exactly why the continuity points have to be watched. Convergence in distribution means \(F_{n}(y)\rightarrow F(y)\) at every continuity point \(y\) of \(F\). Two remarks are used repeatedly. First, \(F\) is non-decreasing, so its discontinuities are at most countable — the open intervals \(\left(F(y^{-}),F(y^{+})\right)\) belonging to distinct jumps are disjoint and each contains a rational — and continuity points are therefore dense. Second, \(F(y)\rightarrow1\) as \(y\rightarrow+\infty\) and \(F(y)\rightarrow0\) as \(y\rightarrow-\infty\).

The sum. Let \(x\) be a continuity point of the distribution function of \(A+b\), that is, let \(F\) be continuous at \(x-b\), and let \(\varepsilon>0\). If \(A_{n}+B_{n}\le x\) and \(\abs{B_{n}-b}\le\varepsilon\) then \(A_{n}\le x-B_{n}\le x-b+\varepsilon\); conversely, if \(A_{n}\le x-b-\varepsilon\) and \(\abs{B_{n}-b}\le\varepsilon\) then \(A_{n}+B_{n}\le x\). Passing to probabilities,

with \(\delta_{n}=\Pr\!\left(\abs{B_{n}-b}>\varepsilon\right)\), which tends to \(0\) because \(B_{n}\rightarrow b\) in probability. Restrict \(\varepsilon\) to the values — all but countably many, by the first remark — for which both \(x-b\pm\varepsilon\) are continuity points of \(F\). Taking the lower and upper limits in Equation (11.42) gives

and letting \(\varepsilon\) decrease to zero through admissible values makes both ends tend to \(F(x-b)\), since \(F\) is continuous at \(x-b\). Hence \(\Pr(A_{n}+B_{n}\le x)\rightarrow F(x-b)=\Pr(A+b\le x)\), which is the claim.

Boundedness in probability. Given \(\delta>0\), choose \(K>0\) with \(\pm K\) both continuity points of \(F\), \(F(K)>1-\delta/2\) and \(F(-K)<\delta/2\); the second remark makes this possible and the first makes it possible at continuity points. Since \(\Pr(\abs{A_{n}}>K)\le\left(1-F_{n}(K)\right)+F_{n}(-K)\),

The product. Decompose \(A_{n}B_{n}=b\,A_{n}+A_{n}\left(B_{n}-b\right)\) and treat the two pieces separately. For the second, fix \(\varepsilon>0\) and \(\delta>0\) and take \(K\) as in Equation (11.43). Since \(\abs{A_{n}(B_{n}-b)}>\varepsilon\) forces either \(\abs{A_{n}}>K\) or \(\abs{B_{n}-b}>\varepsilon/K\),

whose second term tends to \(0\); so the upper limit is at most \(\delta\), and \(\delta\) was arbitrary. Thus \(A_{n}(B_{n}-b)\rightarrow0\) in probability.

For the first piece, \(b\,A_{n}\rightarrow b\,A\) in distribution. This is trivial for \(b=0\), both sides being the constant \(0\). For \(b>0\) the events \(\set{bA_{n}\le x}\) and \(\set{A_{n}\le x/b}\) coincide, and \(x\) is a continuity point of the law of \(bA\) exactly when \(x/b\) is one of \(F\). For \(b<0\) they are \(\set{A_{n}\ge x/b}\), so \(\Pr(bA_{n}\le x)=1-\Pr(A_{n}<x/b)\); and at a continuity point \(y\) of \(F\) one has \(F_{n}(y')\le\Pr(A_{n}<y)\le F_{n}(y)\) for every continuity point \(y'<y\), so the limits squeeze \(\Pr(A_{n}<y)\) between \(F(y')\) and \(F(y)\), and \(F(y')\rightarrow F(y)\) as \(y'\) increases to \(y\). Hence \(\Pr(bA_{n}\le x)\rightarrow1-F(x/b)=\Pr(bA\le x)\) at every such \(x\).

Applying the sum clause already proved, with \(bA_{n}\) in place of \(A_{n}\) and the constant \(0\) in place of \(b\), gives \(A_{n}B_{n}\rightarrow bA\) in distribution.

Derivation of the quotient clause from the product clause. Derives Lemma 11.52. Let \(b\neq0\) and \(\varepsilon>0\). The map \(y\mapsto1/y\) is continuous at \(b\), so there is a \(\delta\in(0,\abs{b}/2)\) such that \(\abs{y-b}<\delta\) implies both \(y\neq0\) and \(\abs{1/y-1/b}<\varepsilon\). Hence

the same bound also showing that \(B_{n}\) is nonzero with probability tending to one, so the quotient is defined on an event of probability approaching \(1\). Thus \(1/B_{n}\rightarrow1/b\) in probability — this is the continuous-mapping theorem, in the only case this chapter needs, and its proof is the two lines just given. Applying the product clause to \(A_{n}\) and \(1/B_{n}\) gives \(A_{n}/B_{n}\rightarrow A/b\) in distribution.

Derivation. Derives Proposition 11.57. For every \(\tau\) the supremum in Equation (11.46) is taken over a subset of \(\Theta\), so \(L^{*}(\tau)\le\sup_{\Theta}L=L(\hat{\theta})\). On the other hand \(\hat{\theta}\) itself lies in the fibre over \(\tau=g(\hat{\theta})\), so \(L^{*}(g(\hat{\theta}))\ge L(\hat{\theta})\). The two inequalities force \(L^{*}(g(\hat{\theta}))=L(\hat{\theta})=\max_{\tau}L^{*}(\tau)\).

Derivation. Derives Lemma 11.61. Because the support does not move with \(\theta\), the domain of the integral below is fixed, and (R4) licenses differentiating under it:

Differentiate that identity once more with respect to \(\theta\), again under the integral:

In the second integral write \(\pp f/\pp\theta=(\pp\ln f/\pp\theta)f\); it becomes \(\avg{U^{2}}=I_{1}\). Rearranging gives Equation (11.48). Finally, for an independent sample \(\ell=\sum_{i}\ln f(x_{i};\theta)\), so the total score is a sum of \(n\) independent copies of \(U\), each of mean zero; variances of independent variables add, giving \(I_{n}=nI_{1}\).

Derivation. Derives Lemma 11.63. Write \(\tilde{V}=V-\avg{V}\) and \(\tilde{W}=W-\avg{W}\), both of mean zero and finite second moment. Their product is integrable, since \(\abs{\tilde{V}\tilde{W}}\le\tfrac12(\tilde{V}^{2}+\tilde{W}^{2})\), so \(\operatorname{cov}(V,W)=\avg{\tilde{V}\tilde{W}}\) is a finite number and

If \(\operatorname{var}V=0\) then \(\tilde{V}=0\) almost surely, both sides of Equation (11.50) vanish, and the stated equality condition holds with \((a,c)=(1,0)\). Otherwise \(\phi\) is a genuine quadratic that is nowhere negative, so its discriminant is non-positive: \(4\operatorname{cov}(V,W)^{2}-4\operatorname{var}V\operatorname{var}W\le0\), which is Equation (11.50). Equality holds exactly when the discriminant vanishes, i.e. when \(\phi\) has a real root \(s_{0}\); and \(\phi(s_{0})=0\) says that the non-negative variable \(\left(s_{0}\tilde{V}+\tilde{W}\right)^{2}\) has zero mean, hence vanishes almost surely, which is the stated condition with \((a,c)=(s_{0},1)\).

Derivation. Derives Theorem 11.64. By Lemma 11.61 the score has mean zero, so \(\operatorname{cov}(T,U)=\avg{TU}-\avg{T}\avg{U}=\avg{TU}\). Evaluate that expectation by the same manoeuvre as before, writing \(U f=\pp f/\pp\theta\):

Now apply Lemma 11.63 to \(T\) and \(U\), whose variances are finite by hypothesis and by Equation (11.52) respectively:

which is Equation (11.53).

Derivation. Derives Corollary 11.65. By Lemma 11.63 equality holds exactly when the two centred variables are proportional with probability one; since \(\operatorname{var}U =I(\theta)>0\) the proportionality can be solved for the score, giving \(U=c(\theta)(T-\avg{T})\) almost surely. Given that for every \(\theta\), integrate \(\pp\ln f/\pp\theta=c(\theta)(T(x)-\avg{T}_{\theta})\) with respect to \(\theta\): the \(x\)-dependence enters only through the factor \(T(x)\) multiplying \(\int c\,\dd\theta\), and the remaining terms are functions of \(\theta\) alone plus a constant of integration \(C(x)\).

Derivation of (ii), given (i). Derives Theorem 11.67. Write \(\ell_{n}(\theta)=\sum_{i}\ln f(X_{i};\theta)\) and expand its derivative about \(\theta_{0}\) with a Lagrange remainder, which (R4) permits:

with \(\bar{\theta}_{n}\) between \(\theta_{0}\) and \(\hat{\theta}_{n}\). Solve for the deviation and multiply by \(\sqrt{n}\):

Treat the three pieces separately.

The numerator is \(n^{-1/2}\sum_{i}U(\theta_{0};X_{i})\), a normalised sum of independent identically distributed variables of mean zero (Lemma 11.61) and variance \(I_{1}(\theta_{0})\), finite and nonzero by (R5). Theorem 11.46 therefore gives \(n^{-1/2}\ell_{n}'(\theta_{0})\rightarrow \mathcal{N}(0,I_{1}(\theta_{0}))\) in distribution.

The first term of the denominator is \(-n^{-1}\sum_{i}\pp^{2}\ln f(X_{i};\theta_{0})/\pp\theta^{2}\), an average of independent identically distributed variables whose common mean is \(I_{1}(\theta_{0})\) by Equation (11.48) — an identity that presupposes the summand integrable, and asserts nothing about its second moment. Nothing in (R1)–(R5) bounds that second moment, so Corollary 11.51, which is the Chebyshev form and needs a finite variance, does not apply here; the weak law in the form that needs only integrability [Billingsley:1995] gives convergence to \(I_{1}(\theta_{0})\) in probability.

The second term of the denominator is bounded in absolute value by \(\tfrac12\abs{\hat{\theta}_{n}-\theta_{0}}\cdot n^{-1}\sum_{i}M(X_{i})\), using the domination in (R5). The average converges in probability to \(\avg{M}<\infty\), and \(\abs{\hat{\theta}_{n}-\theta_{0}}\rightarrow0\) in probability by (i); the product therefore tends to zero in probability.

The two denominator terms therefore converge in probability, by the sum clause of Lemma 11.52, to the constant \(I_{1}(\theta_{0})\), which is nonzero by (R5). Equation (11.57) is a quotient, not a sum or a product, so the passage to the limit is the quotient clause of Lemma 11.52 — available exactly because \(I_{1}(\theta_{0})\neq0\), and resting on the convergence \(1/B_{n}\rightarrow1/I_{1}(\theta_{0})\) in probability established at Equation (11.44). The ratio therefore converges in distribution to \(\mathcal{N}(0,I_{1}(\theta_{0}))/I_{1}(\theta_{0}) =\mathcal{N}(0,I_{1}(\theta_{0})^{-1})\), since scaling a Gaussian by \(1/I_{1}\) scales its variance by \(1/I_{1}^{2}\).

Derivation. Derives Lemma 11.68. Put \(a=\avg{Y}\), which is finite and strictly positive because \(Y>0\). The logarithm has second derivative \(-1/y^{2}<0\) on \((0,\infty)\), so it is strictly concave there and its graph lies below the tangent at \(a\):

with equality only at \(y=a\). To see Equation (11.59) without appealing to a general theory of convexity, apply the mean value theorem (Theorem 7.35) to \(h(y)=\ln y-\ln a-(y-a)/a\): \(h(a)=0\) and \(h'(y)=1/y-1/a\), which is positive for \(y<a\) and negative for \(y>a\), so \(h\) increases up to \(a\) and decreases after it, attaining its strict maximum \(0\) there. Now put \(y=Y\) in Equation (11.59) and take expectations, which is legitimate because the right-hand side is integrable:

Equality forces the non-positive variable \(\ln Y-\ln a-(Y-a)/a\) to have zero mean, hence to vanish almost surely, which by the strictness in Equation (11.59) means \(Y=a\) almost surely.

Derivation of (i). Derives Theorem 11.67. Take \(k=1\); the modification for \(k\) parameters is given at the end. Write \(I=I_{1}(\theta_{0})\), which is finite and strictly positive by (R5), and let \(a_{0}>0\) be small enough that \([\theta_{0}-a_{0},\theta_{0}+a_{0}]\) lies both inside \(\Theta\), which (R3) permits, and inside the neighbourhood on which (R5) supplies the dominating function \(M\). For \(0<a\le a_{0}\) and \(\theta=\theta_{0}\pm a\), expand with a Lagrange remainder, as (R4) allows:

with \(\bar{\theta}\) between \(\theta_{0}\) and \(\theta\).

Each of the three coefficients is an average of independent identically distributed terms. The first, \(n^{-1}\sum_{i}U(\theta_{0};X_{i})\), converges in probability to \(0\), the score having mean zero by Lemma 11.61; the second converges in probability to \(-I\) by Equation (11.48); and the third is bounded in absolute value by \(n^{-1}\sum_{i}M(X_{i})\), which converges in probability to \(\bar{M}=\avg{M}<\infty\). Corollary 11.51 gives the first of these, the score having variance \(I_{1}(\theta_{0})\) by (R5). It gives neither of the other two: (R5) bounds the variance of \(\pp^{2}\ln f/\pp\theta^{2}\) no more than it bounds that of \(M\), which it assumes integrable and nothing more, and Equation (11.48) says only that the second derivative has a mean. Both are averages of integrable summands, so for both the weak law in the form that needs only integrability is used instead [Billingsley:1995].

Now shrink the radius, requiring in addition

and let \(E_{n}(a)\) be the event on which all three of

hold. By the previous paragraph \(\Pr(E_{n}(a))\rightarrow1\). On \(E_{n}(a)\), and for either sign of \(\theta-\theta_{0}=\pm a\), the three terms of Equation (11.61) are bounded by \(a^{2}I/16\), by \(-a^{2}I/2+a^{2}I/8\) and, using Equation (11.62), by \(a^{3}(\bar{M}+1)/6<a^{2}I/6\) respectively, so that

From Equations (11.61) and (11.62) (bound the three terms on the event \(E_{n}(a)\) and add). So on \(E_{n}(a)\) the log-likelihood is strictly smaller at both endpoints of \([\theta_{0}-a,\theta_{0}+a]\) than at its midpoint. It is continuous there, so by the extreme value theorem (Theorem 7.24) it attains a maximum on that interval; the maximum is at neither endpoint, hence at an interior point, where the derivative of a differentiable function vanishes. The score equation \(\ell_{n}'(\theta)=0\) therefore has a solution in \((\theta_{0}-a,\theta_{0}+a)\).

Define \(\hat{\theta}_{n}\) to be a solution of the score equation in \([\theta_{0}-a_{0},\theta_{0}+a_{0}]\) nearest to \(\theta_{0}\) when one exists — the solution set is closed, being the zero set of a continuous function, so a nearest point is attained — and any fixed point of \(\Theta\) otherwise. For every \(a\) satisfying Equation (11.62), \(E_{n}(a)\) implies \(\abs{\hat{\theta}_{n}-\theta_{0}}<a\), so \(\Pr(\abs{\hat{\theta}_{n}-\theta_{0}}<a)\rightarrow1\); and since the event grows with \(a\), the same holds for every \(a>0\). That is convergence in probability.

In \(k\) parameters, replace the interval by the closed ball of radius \(a\) about \(\theta_{0}\) and its endpoints by the sphere, expanding along \(\theta=\theta_{0}+a\vect{e}\) with \(\vect{e}\) a unit vector. The linear term is bounded by \(a\) times the length of \(n^{-1}\nabla\ell_{n}\); the quadratic term is \(-\tfrac12a^{2}\,\vect{e}\transpose I\vect{e}\) up to the same error; and the cubic term is bounded as before, with \(M\) dominating every third partial derivative. Read in \(k\) parameters, (R5) makes \(I\) positive definite, and \(\vect{e}\mapsto\vect{e}\transpose I\vect{e}\) is continuous and strictly positive on the unit sphere, which is closed and bounded, so it has a strictly positive minimum \(\lambda\) (Theorem 7.24); putting \(\lambda\) in place of \(I\) throughout gives Equation (11.63) at every point of the sphere, and the argument closes as before with the gradient in place of the derivative.

Derivation. Derives Proposition 11.71. \(L_{\mathrm{p}}\) is the induced likelihood Equation (11.46) for the map \(g(\mu,\vect{\nu})=\mu\), so Proposition 11.57 applies verbatim: \(g\) of the unconstrained maximiser, namely \(\hat{\mu}\), maximises \(L_{\mathrm{p}}\), and the maximum value is \(L(\hat{\mu},\hat{\vect{\nu}})\). The ratio is therefore at most \(1\) and equals \(1\) at \(\hat{\mu}\); it is positive because \(L_{\mathrm{p}}(\mu)\ge L(\mu,\vect{\nu})>0\) for any admissible \(\vect{\nu}\).

Derivation. Derives Proposition 11.72. By Theorem 11.67 the asymptotic covariance of \(\sqrt{n}(\hat{\theta}-\theta_{0})\) is \(I^{-1}\), so the asymptotic variance of \(\sqrt{n}(\hat{\mu}-\mu_{0})\) is the \(\mu\mu\) entry of \(I^{-1}\). To identify it, solve \(I\,(x,\vect{y})\transpose=(1,\vect{0})\transpose\): the lower block gives \(I_{\nu\mu}x+I_{\nu\nu}\vect{y}=\vect{0}\), hence \(\vect{y}=-I_{\nu\nu}^{-1}I_{\nu\mu}x\), and substituting in the upper block gives \(\left(I_{\mu\mu}-I_{\mu\nu}I_{\nu\nu}^{-1}I_{\nu\mu}\right)x=1\). By construction \(x=\left(I^{-1}\right)_{\mu\mu}\), so \(\left(I^{-1}\right)_{\mu\mu}=I_{\mathrm{p}}^{-1}\). Finally \(I_{\mu\nu}I_{\nu\nu}^{-1}I_{\nu\mu}\ge0\) because \(I_{\nu\nu}^{-1}\) is positive definite, and it vanishes exactly when \(I_{\mu\nu}=0\).

Derivation. Derives Proposition 11.76. For \(\alpha\in(0,1)\), \(G\) is invertible on its range, and \(G\) is decreasing, so \(\set{G(t)\le\alpha}=\set{t\ge G^{-1}(\alpha)}\). Hence \(\Pr(p\le\alpha\mid H_{0})=\Pr(t\ge G^{-1}(\alpha)\mid H_{0}) =G(G^{-1}(\alpha))=\alpha\), which is the distribution function of the uniform law on \([0,1]\).

Derivation. Derives Theorem 11.78. Consider the function

It is non-negative everywhere. Where \(f_{1}>kf_{0}\) the second factor is positive and \(\psi^{*}=1\ge\psi\), so the first is non-negative; where \(f_{1}<kf_{0}\) the second factor is negative and \(\psi^{*}=0\le\psi\), so the first is non-positive; where \(f_{1}=kf_{0}\) the second factor vanishes. Integrating,

so that \(\avg{\psi^{*}}_{1}-\avg{\psi}_{1}\ge k\left(\alpha-\avg{\psi}_{0}\right)\). The right-hand side is non-negative because \(k\ge0\) and \(\avg{\psi}_{0}\le\alpha\) by hypothesis.

Derivation. Derives Proposition 11.81. Work with the Laplace transform \(M_{k}(s)=\avg{\ee^{-sX}}\), which for a distribution on \((0,\infty)\) exists for all \(s\ge0\). For one degree of freedom, completing the exponent,

using \(\int\ee^{-au^{2}/2}\dd u=\sqrt{2\pi/a}\). The \(k\) squares are independent, so their transforms multiply exactly as characteristic functions do in Proposition 11.43(iii), giving \(M_{k}(s)=(1+2s)^{-k/2}\).

Now transform the claimed density. Substituting \(y=(\tfrac12+s)x\),

the middle integral being \(\Gamma(k/2)\) by definition. The two distributions have the same Laplace transform on \(s>0\); since both are supported on \([0,\infty)\) their transforms extend analytically to \(\operatorname{Re}s>-\tfrac12\) and agree there, in particular on the imaginary axis, where they are the characteristic functions of Definition 11.42. Uniqueness of the characteristic function [Billingsley:1995] [Feller:1971] then forces the distributions to coincide, which is Equation (11.73).

For the moments, differentiate \(M_{k}\) at \(s=0\): \(M_{k}'(s)=-k(1+2s)^{-k/2-1}\) and \(M_{k}''(s)=k(k+2)(1+2s)^{-k/2-2}\), so \(\avg{X}=-M_{k}'(0)=k\) and \(\avg{X^{2}}=M_{k}''(0)=k(k+2)\), whence \(\operatorname{var}X=k(k+2)-k^{2}=2k\).

Derivation. Derives Corollary 11.82. \(\chi^{2}_{1}=Z^{2}\) with \(Z\) standard Gaussian, and \(Z^{2}\ge z^{2}\) exactly when \(\abs{Z}\ge z\), an event of probability \(\Pr(Z\ge z)+\Pr(Z\le-z)=2(1-\Phi(z))\) by symmetry.

Derivation. Derives Lemma 11.83. By the spectral theorem for real symmetric matrices — diagonalise \(\Sigma\) by an orthogonal matrix and take the positive square roots of its eigenvalues, which are positive because \(\Sigma\) is positive definite — \(\Sigma\) has a symmetric positive-definite square root \(\Sigma^{1/2}\). Put \(\vect{V}=\Sigma^{-1/2}\vect{Y}\); a linear image of a Gaussian vector is Gaussian, with covariance \(\Sigma^{-1/2}\Sigma\,\Sigma^{-1/2}=\identity\), so the components of \(\vect{V}\) are independent standard Gaussians. Then \(\vect{Y}\transpose\Sigma^{-1}\vect{Y}=\vect{V}\transpose\vect{V} =\sum_{i}V_{i}^{2}\), which is \(\chi^{2}_{k}\) by Definition 11.80. For the second statement choose an orthonormal basis \(\vect{u}_{1},\dots,\vect{u}_{r}\) of the image of \(P\); the components \(\vect{u}_{j}\cdot\vect{V}\) are jointly Gaussian with \(\avg{(\vect{u}_{i}\cdot\vect{V})(\vect{u}_{j}\cdot\vect{V})} =\vect{u}_{i}\cdot\vect{u}_{j}=\delta_{ij}\), hence independent standard Gaussians, and \(\norm{P\vect{V}}^{2}=\sum_{j=1}^{r} (\vect{u}_{j}\cdot\vect{V})^{2}\).

Derivation. Derives Corollary 11.84. Put \(V_{i}=(x_{i}-\mu)/u_{i}\), so that \(\vect{V}\in\R^{n}\) is Gaussian with mean zero and covariance \(\identity\). Write \(S=\sum_{k}u_{k}^{-2}\) and let \(\vect{m}\) be the vector with components \(m_{i}=u_{i}^{-1}/\sqrt{S}\), which is a unit vector because \(\sum_{i}m_{i}^{2}=S^{-1}\sum_{i}u_{i}^{-2}=1\). From Equation (11.76),

and therefore

where \(P=\identity-\vect{m}\vect{m}\transpose\) is the orthogonal projector onto the hyperplane orthogonal to \(\vect{m}\), of dimension \(n-1\). Hence \(\chi^{2}=\norm{P\vect{V}}^{2}\), which is \(\chi^{2}_{n-1}\) by the second statement of Lemma 11.83; \(\mu\) has cancelled, as Equation (11.79) shows. The mean and variance are Proposition 11.81 with \(k=n-1\), and the rescaling claim is immediate because \(\chi^{2}\) is homogeneous of degree \(-2\) in the uncertainties.

Derivation, one parameter and a simple null. Derives Theorem 11.88. Take \(k=r=1\) and \(\Theta_{0}=\set{\theta_{0}}\). Expand \(\ell_{n}\) about the unconstrained MLE \(\hat{\theta}_{n}\), where \(\ell_{n}'\) vanishes:

with \(\bar{\theta}_{n}\) between the two. Multiply by \(-2\):

From Equation (11.82) (multiply by \(-2\) and rescale the deviation by \(\sqrt{n}\)). The first bracket converges in distribution to \(\mathcal{N}(0,I_{1}(\theta_{0})^{-1})\) by Theorem 11.67, and is therefore bounded in probability. The second factor converges in probability to \(I_{1}(\theta_{0})\): it is an average of independent identically distributed terms evaluated at \(\hat{\theta}_{n}\) rather than at \(\theta_{0}\), and the difference is controlled by the domination in (R5) together with the consistency of \(\hat{\theta}_{n}\). The last term is bounded in absolute value by \(\tfrac13\left(n^{-1}\sum_{i}M(X_{i})\right) n\abs{\hat{\theta}_{n}-\theta_{0}}^{3}\), and \(n\abs{\hat{\theta}_{n}-\theta_{0}}^{3} =n^{-1/2}\left[\sqrt{n}\abs{\hat{\theta}_{n}-\theta_{0}}\right]^{3} \rightarrow0\) in probability. By Lemma 11.52,

and \(\sqrt{I_{1}}\,W\) is standard Gaussian, so its square is \(\chi^{2}_{1}\) by Definition 11.80.

Derivation, \(k\) parameters and \(r\) constraints. Derives Theorem 11.88. The same expansion in \(k\) dimensions gives, for any \(\theta\) near \(\hat{\theta}_{n}\),

the Hessian term being handled exactly as above and the cubic remainder again vanishing. Since \(\hat{\theta}_{0}\) maximises \(\ell_{n}\) over \(\Theta_{0}\), it minimises \(Q\) there, and \(q=Q(\hat{\theta}_{0})=\min_{\theta\in\Theta_{0}}Q(\theta)\). Near \(\theta_{0}\) the submanifold \(\Theta_{0}\) is its tangent space to the accuracy the expansion retains, \(\Theta_{0}\simeq\theta_{0}+E\) with \(E\) that tangent space, of dimension \(\dim E=k-r\). Writing \(\vect{W}=\sqrt{n}\left(\hat{\theta}_{n}-\theta_{0}\right)\) and \(I=I_{1}(\theta_{0})\),

Substitute \(\vect{V}=I^{1/2}\vect{W}\) and \(S=I^{1/2}E\); the right side is \(\min_{\vect{s}\in S}\norm{\vect{V}-\vect{s}}^{2} =\norm{P\vect{V}}^{2}\) with \(P\) the orthogonal projector onto \(S^{\perp}\), of dimension \(r\). By Theorem 11.67 \(\vect{W}\rightarrow\mathcal{N}(0,I^{-1})\), so \(\vect{V}\) is asymptotically Gaussian with covariance \(I^{1/2}I^{-1}I^{1/2}=\identity\), and Lemma 11.83 gives \(\norm{P\vect{V}}^{2}\sim\chi^{2}_{r}\).

Derivation. Derives Corollary 11.89. The null constrains one coordinate and leaves the other \(m\) free, so \(\Theta_{0}\) is a submanifold of codimension \(r=1\) and Theorem 11.88 applies with \(r=1\). The nuisance parameters affect the variance of \(\hat{\mu}\), through the Schur complement Equation (11.67), but not the number of degrees of freedom.

Derivation. Derives Proposition 11.93. The mixture is the one derived in Equation (11.86): writing \(q_{0}=\hat{\mu}^{2}/\sigma^{2}\) for the two-sided statistic, which is asymptotically \(\chi^{2}_{1}\) by Corollary 11.89, the event \(\tilde{q}_{0}\ge q\) with \(q>0\) is the event \(\set{\hat{\mu}>0}\cap\set{q_{0}\ge q}\). The sign of \(\hat{\mu}\) and the magnitude \(\abs{\hat{\mu}}\) are independent for a Gaussian of mean zero, and the sign is positive with probability \(\tfrac12\), so \(\Pr(\tilde{q}_{0}\ge q)=\tfrac12\Pr(\chi^{2}_{1}\ge q)\). Insert Equation (11.75) with \(z=\sqrt{q}\): the right-hand side is \(\tfrac12\cdot2\left(1-\Phi(\sqrt{q})\right)=1-\Phi(\sqrt{q})\). Comparing with Equation (11.87) gives \(Z=\sqrt{q}\).

Derivation. Derives Proposition 11.97. For any fixed \(m_{\mathrm{fix}}\in M\) the event \(\set{\tilde{q}(m_{\mathrm{fix}})\ge u}\) is contained in the event \(\set{\max_{m\in M}\tilde{q}(m)\ge u}\); a probability is monotone under inclusion.

Derivation. Derives Proposition 11.98. Write \(A_{j}=\set{\tilde{q}(m_{j})\ge u}\). The first claim is Boole's inequality \(\Pr(\bigcup_{j}A_{j})\le\sum_{j}\Pr(A_{j})\), which follows by induction from \(\Pr(A\cup B)=\Pr(A)+\Pr(B)-\Pr(A\cap B)\le\Pr(A)+\Pr(B)\), together with the fact that each \(\Pr(A_{j})\) equals \(p_{\mathrm{loc}}\) because the statistic has the same null distribution at every \(m\). If the \(A_{j}\) are independent then so are their complements, so \(\Pr(\bigcap_{j}A_{j}^{c})=\left(1-p_{\mathrm{loc}}\right)^{N}\); expanding by the binomial theorem gives the series.

Derivation. Derives Theorem 11.101. Suppose \(\max_{m\in M}\tilde{q}(m)\ge u\) and that the left endpoint is below the level, \(\tilde{q}(m_{1})<u\). The path is continuous and reaches a value at least \(u\) somewhere in \(M\), so by the intermediate value theorem (Theorem 7.23) it attains the value \(u\) at some interior point, having come from below; that is an upcrossing, so \(N_{u}\ge1\). Hence

and subadditivity gives \(\Pr(\max\ge u)\le\Pr(\tilde{q}(m_{1})\ge u)+\Pr(N_{u}\ge1)\). Finally \(N_{u}\) is non-negative, so Markov's inequality (Lemma 11.50) with \(a=1\) gives \(\Pr(N_{u}\ge1)\le\avg{N_{u}}\).

Derivation. Derives Theorem 11.103. Fix \(u>0\). An upcrossing of the level \(u\) by \(\tilde{q}\) is exactly an upcrossing of the level \(\sqrt{u}\) by \(X\). Where \(X\le0\) the statistic takes the value \(0\), which is below \(u\), so no crossing of \(u\) occurs there; and on the open set where \(X>0\) one has \(\tilde{q}=X^{2}\), a strictly increasing function of \(X\), so \(\tilde{q}\) passes from below \(u\) to above it precisely where \(X\) passes from below \(\sqrt{u}\) to above it. The two counts therefore agree path by path, \(N_{u}=N_{\sqrt{u}}(X)\), and taking expectations and applying Equation (11.95) at \(z=\sqrt{u}\),

where the integral does not depend on \(u\). Writing the same identity at \(u_{0}\) and dividing cancels the integral and leaves Equation (11.96).

Derivation. Derives Corollary 11.105. Divide Equation (11.94) by \(p_{\mathrm{loc}}\) and use Equations (11.90) and (11.96) for the first form. For the second, the Gaussian tail satisfies

so that \(1-\Phi(z)=\ee^{-z^{2}/2}\left(1+O(z^{-2})\right) /\left(z\sqrt{2\pi}\right)\). To see Equation (11.99), write \(J=\int_{z}^{\infty}\ee^{-t^{2}/2}\dd t\) and integrate by parts with \(\ee^{-t^{2}/2}=t^{-1}\cdot t\,\ee^{-t^{2}/2}\):

The subtracted integral is positive, giving the upper bound, and is at most \(z^{-2}J\), giving \(J(1+z^{-2})\ge\ee^{-z^{2}/2}/z\) and hence the lower bound. Now put \(z=\sqrt{u}=Z_{\mathrm{loc}}\) in the first form of Equation (11.98): the exponentials \(\ee^{-u/2}\) and \(\ee^{-z^{2}/2}\) cancel and what remains is \(\avg{N_{u_{0}}}\ee^{u_{0}/2}\,Z_{\mathrm{loc}}\sqrt{2\pi}\).