1 凸損失の改善 定理 1.1 (Rao–Blackwell の定理). ( X , A ) (\mathcal X,\mathcal A) ( X , A ) ( P θ ) θ ∈ Θ (P_\theta)_{\theta\in\Theta} ( P θ ) θ ∈ Θ T : ( X , A ) → ( Y , B ) T:(\mathcal X,\mathcal A)\to(\mathcal Y,\mathcal B) T : ( X , A ) → ( Y , B ) A ⊆ R A\subseteq\R A ⊆ R δ : X → A \delta:\mathcal X\to A δ : X → A θ ∈ Θ \theta\in\Theta θ ∈ Θ E θ ∣ δ ∣ < ∞ E_\theta|\delta|<\infty E θ ∣ δ ∣ < ∞
各θ ∈ Θ \theta\in\Theta θ ∈ Θ L θ : A → [ 0 , ∞ ) L_\theta:A\to[0,\infty) L θ : A → [ 0 , ∞ ) L θ ( δ ) L_\theta(\delta) L θ ( δ ) P θ P_\theta P θ θ \theta θ B \mathcal B B g : Y → A g:\mathcal Y\to A g : Y → A
δ T = g ( T ) \delta_T=g(T) δ T = g ( T ) は、すべてのθ ∈ Θ \theta\in\Theta θ ∈ Θ
δ T = E θ [ δ ∣ σ ( T ) ] P θ -ほとんど確実に \delta_T=E_\theta[\delta\mid\sigma(T)]\qquad P_\theta\text{-ほとんど確実に} δ T = E θ [ δ ∣ σ ( T )] P θ - ほとんど確実に を満たす。さらに、L θ ( δ T ) L_\theta(\delta_T) L θ ( δ T ) P θ P_\theta P θ
R θ ( δ T ) : = E θ [ L θ ( δ T ) ] ≤ E θ [ L θ ( δ ) ] = : R θ ( δ ) R_\theta(\delta_T):=E_\theta[L_\theta(\delta_T)]
\leq E_\theta[L_\theta(\delta)]=:R_\theta(\delta) R θ ( δ T ) := E θ [ L θ ( δ T )] ≤ E θ [ L θ ( δ )] =: R θ ( δ ) がすべてのθ ∈ Θ \theta\in\Theta θ ∈ Θ
証明. §E14.5 命題 3.3 をu = δ u=\delta u = δ B \mathcal B B a δ a_\delta a δ θ ∈ Θ \theta\in\Theta θ ∈ Θ
a δ ( T ) = E θ [ δ ∣ σ ( T ) ] P θ -ほとんど確実に a_\delta(T)=E_\theta[\delta\mid\sigma(T)]\qquad P_\theta\text{-ほとんど確実に} a δ ( T ) = E θ [ δ ∣ σ ( T )] P θ - ほとんど確実に となるように取る。a 0 ∈ A a_0\in A a 0 ∈ A
g ( y ) = { a δ ( y ) , a δ ( y ) ∈ A , a 0 , a δ ( y ) ∉ A g(y)=
\begin{cases}
a_\delta(y),&a_\delta(y)\in A,\\
a_0,&a_\delta(y)\notin A
\end{cases} g ( y ) = { a δ ( y ) , a 0 , a δ ( y ) ∈ A , a δ ( y ) ∈ / A と定める。区間A A A g g g B \mathcal B B A A A
θ ∈ Θ \theta\in\Theta θ ∈ Θ M = a δ ( T ) M=a_\delta(T) M = a δ ( T ) α = inf A \alpha=\inf A α = inf A δ ≥ α \delta\geq\alpha δ ≥ α §E11.11 命題 2.1 (2) からM ≥ α M\geq\alpha M ≥ α P θ P_\theta P θ α ∉ A \alpha\notin A α ∈ / A δ − α > 0 \delta-\alpha>0 δ − α > 0 B = { M = α } ∈ σ ( T ) B=\{M=\alpha\}\in\sigma(T) B = { M = α } ∈ σ ( T )
E θ [ ( δ − α ) 1 B ] = E θ [ ( M − α ) 1 B ] = 0 E_\theta[(\delta-\alpha)\mathbf1_B]
=E_\theta[(M-\alpha)\mathbf1_B]=0 E θ [( δ − α ) 1 B ] = E θ [( M − α ) 1 B ] = 0 となるため、P θ ( B ) = 0 P_\theta(B)=0 P θ ( B ) = 0 A A A M > α M>\alpha M > α P θ P_\theta P θ β = sup A \beta=\sup A β = sup A β − δ \beta-\delta β − δ M ≤ β M\leq\beta M ≤ β β ∉ A \beta\notin A β ∈ / A M < β M<\beta M < β M ∈ A M\in A M ∈ A P θ P_\theta P θ
関数g g g a δ a_\delta a δ A A A
δ T = g ( T ) = M = E θ [ δ ∣ σ ( T ) ] \delta_T=g(T)=M=E_\theta[\delta\mid\sigma(T)] δ T = g ( T ) = M = E θ [ δ ∣ σ ( T )] がP θ P_\theta P θ §E11.11 定理 3.2 を区間A A A L θ L_\theta L θ
L θ ( δ T ) ≤ E θ [ L θ ( δ ) ∣ σ ( T ) ] P θ -ほとんど確実に L_\theta(\delta_T)
\leq E_\theta[L_\theta(\delta)\mid\sigma(T)]
\qquad P_\theta\text{-ほとんど確実に} L θ ( δ T ) ≤ E θ [ L θ ( δ ) ∣ σ ( T )] P θ - ほとんど確実に を得る。右辺は非負かつ可積分であるから、左辺も可積分である。両辺の期待値を取り、期待値保存§E11.11 命題 2.1 (4) を用いると、
R θ ( δ T ) ≤ R θ ( δ ) R_\theta(\delta_T)\leq R_\theta(\delta) R θ ( δ T ) ≤ R θ ( δ ) となる。g g g θ \theta θ θ ∈ Θ \theta\in\Theta θ ∈ Θ ▨
例 1.2 (凸性を外した場合). P ( X = 1 ) = P ( X = − 1 ) = 1 / 2 P(X=1)=P(X=-1)=1/2 P ( X = 1 ) = P ( X = − 1 ) = 1/2 T = 0 T=0 T = 0 δ = X \delta=X δ = X
E [ δ ∣ σ ( T ) ] = E [ X ] = 0 E[\delta\mid\sigma(T)]=E[X]=0 E [ δ ∣ σ ( T )] = E [ X ] = 0 である。行動空間をA = [ − 1 , 1 ] A=[-1,1] A = [ − 1 , 1 ] L ( a ) = 1 − a 2 L(a)=1-a^2 L ( a ) = 1 − a 2 L L L
E [ L ( δ ) ] = 0 , E [ L ( E [ δ ∣ σ ( T ) ] ) ] = L ( 0 ) = 1 E[L(\delta)]=0,
\qquad
E[L(E[\delta\mid\sigma(T)])]=L(0)=1 E [ L ( δ )] = 0 , E [ L ( E [ δ ∣ σ ( T )])] = L ( 0 ) = 1 である。したがって、損失の凸性を外すと、十分統計量による条件付けがリスクを増加させる場合がある。
2 不偏性と分散 定理 2.1. ( X , A ) (\mathcal X,\mathcal A) ( X , A ) ( P θ ) θ ∈ Θ (P_\theta)_{\theta\in\Theta} ( P θ ) θ ∈ Θ T : ( X , A ) → ( Y , B ) T:(\mathcal X,\mathcal A)\to(\mathcal Y,\mathcal B) T : ( X , A ) → ( Y , B ) τ : Θ → R \tau:\Theta\to\R τ : Θ → R τ ^ \widehat\tau τ θ ∈ Θ \theta\in\Theta θ ∈ Θ
E θ [ τ ^ ] = τ ( θ ) , E θ [ τ ^ 2 ] < ∞ E_\theta[\widehat\tau]=\tau(\theta),
\qquad E_\theta[\widehat\tau^2]<\infty E θ [ τ ] = τ ( θ ) , E θ [ τ 2 ] < ∞ を満たすとする。このとき有限値B \mathcal B B h h h τ ~ = h ( T ) \widetilde\tau=h(T) τ = h ( T )
τ ~ \widetilde\tau τ θ \theta θ τ ~ \widetilde\tau τ τ ( θ ) \tau(\theta) τ ( θ ) θ ∈ Θ \theta\in\Theta θ ∈ Θ E θ [ τ ~ ] = τ ( θ ) E_\theta[\widetilde\tau]=\tau(\theta) E θ [ τ ] = τ ( θ ) すべてのθ ∈ Θ \theta\in\Theta θ ∈ Θ Var θ ( τ ^ ) − Var θ ( τ ~ ) = E θ [ ( τ ^ − τ ~ ) 2 ] ≥ 0 \operatorname{Var}_\theta(\widehat\tau)-\operatorname{Var}_\theta(\widetilde\tau)
=E_\theta[(\widehat\tau-\widetilde\tau)^2]\geq0 Var θ ( τ ) − Var θ ( τ ) = E θ [( τ − τ ) 2 ] ≥ 0 θ \theta θ τ ^ = τ ~ \widehat\tau=\widetilde\tau τ = τ P θ P_\theta P θ τ ^ \widehat\tau τ P θ P_\theta P θ σ ( T ) \sigma(T) σ ( T )
証明. 定理 1.1 をA = R A=\R A = R δ = τ ^ \delta=\widehat\tau δ = τ
L θ ( a ) = ( a − τ ( θ ) ) 2 L_\theta(a)=(a-\tau(\theta))^2 L θ ( a ) = ( a − τ ( θ ) ) 2 に適用する。E θ [ τ ^ 2 ] < ∞ E_\theta[\widehat\tau^2]<\infty E θ [ τ 2 ] < ∞ L θ ( τ ^ ) L_\theta(\widehat\tau) L θ ( τ ) B \mathcal B B h h h τ ~ = h ( T ) \widetilde\tau=h(T) τ = h ( T )
τ ~ = E θ [ τ ^ ∣ σ ( T ) ] P θ -ほとんど確実に \widetilde\tau=E_\theta[\widehat\tau\mid\sigma(T)]
\qquad P_\theta\text{-ほとんど確実に} τ = E θ [ τ ∣ σ ( T )] P θ - ほとんど確実に がすべてのθ ∈ Θ \theta\in\Theta θ ∈ Θ T T T h h h (1) が成り立つ。
(2) を示す。期待値保存§E11.11 命題 2.1 (4) により、任意のθ ∈ Θ \theta\in\Theta θ ∈ Θ
E θ [ τ ~ ] = E θ [ τ ^ ] = τ ( θ ) E_\theta[\widetilde\tau]=E_\theta[\widehat\tau]=\tau(\theta) E θ [ τ ] = E θ [ τ ] = τ ( θ ) である。
(3) を示す。θ ∈ Θ \theta\in\Theta θ ∈ Θ §E11.11 定理 4.4 とτ ~ = E θ [ τ ^ ∣ σ ( T ) ] \widetilde\tau=E_\theta[\widehat\tau\mid\sigma(T)] τ = E θ [ τ ∣ σ ( T )]
Var θ ( τ ^ ) = E θ [ ( τ ^ − τ ~ ) 2 ] + Var θ ( τ ~ ) \operatorname{Var}_\theta(\widehat\tau)
=E_\theta[(\widehat\tau-\widetilde\tau)^2]
+\operatorname{Var}_\theta(\widetilde\tau) Var θ ( τ ) = E θ [( τ − τ ) 2 ] + Var θ ( τ ) を得る。したがって分散差の恒等式と分散改善が成り立つ。分散が等しいことは、非負確率変数( τ ^ − τ ~ ) 2 (\widehat\tau-\widetilde\tau)^2 ( τ − τ ) 2 τ ^ = τ ~ \widehat\tau=\widetilde\tau τ = τ P θ P_\theta P θ τ ~ \widetilde\tau τ σ ( T ) \sigma(T) σ ( T ) τ ^ \widehat\tau τ σ ( T ) \sigma(T) σ ( T ) τ ^ \widehat\tau τ σ ( T ) \sigma(T) σ ( T ) §E11.11 命題 2.1 (3) によりτ ^ = τ ~ \widehat\tau=\widetilde\tau τ = τ P θ P_\theta P θ ▨
3 Bernoulli 標本 例 3.1. n ≥ 2 n\geq2 n ≥ 2 X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n p ∈ ( 0 , 1 ) p\in(0,1) p ∈ ( 0 , 1 ) X 1 X_1 X 1 p p p §E14.5 例 1.3 により成功回数
T = ∑ i = 1 n X i T=\sum_{i=1}^nX_i T = i = 1 ∑ n X i はp p p
s ∈ { 1 , … , n − 1 } s\in\{1,\ldots,n-1\} s ∈ { 1 , … , n − 1 } T = s T=s T = s ( n − 1 s − 1 ) \binom{n-1}{s-1} ( s − 1 n − 1 )
E p [ X 1 ∣ T = s ] = ( n − 1 s − 1 ) ( n s ) = s n E_p[X_1\mid T=s]
=\frac{\binom{n-1}{s-1}}{\binom ns}
=\frac sn E p [ X 1 ∣ T = s ] = ( s n ) ( s − 1 n − 1 ) = n s となる。s = 0 , n s=0,n s = 0 , n
E p [ X 1 ∣ σ ( T ) ] = T n = X ‾ E_p[X_1\mid\sigma(T)]=\frac Tn=\overline X E p [ X 1 ∣ σ ( T )] = n T = X であり、条件付き期待値の表示はp p p
T ∼ Bin ( n , p ) T\sim\operatorname{Bin}(n,p) T ∼ Bin ( n , p ) §E11.5 命題 1.3 により
Var p ( X 1 ) = p ( 1 − p ) , Var p ( X ‾ ) = p ( 1 − p ) n . \operatorname{Var}_p(X_1)=p(1-p),
\qquad
\operatorname{Var}_p(\overline X)=\frac{p(1-p)}n. Var p ( X 1 ) = p ( 1 − p ) , Var p ( X ) = n p ( 1 − p ) . 二乗損失L p ( a ) = ( a − p ) 2 L_p(a)=(a-p)^2 L p ( a ) = ( a − p ) 2 p ∈ ( 0 , 1 ) p\in(0,1) p ∈ ( 0 , 1 ) n ≥ 2 n\geq2 n ≥ 2 X ‾ \overline X X X 1 X_1 X 1
4 Jensen の不等式の等号 命題 4.1 (Jensen の不等式と狭義凸の場合の等号). I ⊆ R I\subseteq\R I ⊆ R φ : I → R \varphi:I\to\R φ : I → R X X X φ ( X ) \varphi(X) φ ( X ) P ( X ∈ I ) = 1 P(X\in I)=1 P ( X ∈ I ) = 1 E [ X ] ∈ I E[X]\in I E [ X ] ∈ I
φ \varphi φ φ ( E [ X ] ) ≤ E [ φ ( X ) ] \varphi(E[X])\leq E[\varphi(X)] φ ( E [ X ]) ≤ E [ φ ( X )] φ \varphi φ φ ( E [ X ] ) = E [ φ ( X ) ] \varphi(E[X])=E[\varphi(X)] φ ( E [ X ]) = E [ φ ( X )] X = E [ X ] X=E[X] X = E [ X ] φ \varphi φ φ ( E [ X ] ) ≥ E [ φ ( X ) ] \varphi(E[X])\geq E[\varphi(X)] φ ( E [ X ]) ≥ E [ φ ( X )] φ \varphi φ 証明. 凸関数に対する不等式は§E11.4 定理 3.4 である。m = E [ X ] m=E[X] m = E [ X ] φ \varphi φ m m m I I I X − m X-m X − m m − X m-X m − X X = m X=m X = m
m m m I I I m m m s s s
D = φ ( X ) − φ ( m ) − s ( X − m ) D=\varphi(X)-\varphi(m)-s(X-m) D = φ ( X ) − φ ( m ) − s ( X − m ) と置く。支持直線の性質からD ≥ 0 D\geq0 D ≥ 0 D = 0 D=0 D = 0 X = m X=m X = m
E [ D ] = E [ φ ( X ) ] − φ ( m ) − s ( E [ X ] − m ) = 0 E[D]=E[\varphi(X)]-\varphi(m)-s(E[X]-m)=0 E [ D ] = E [ φ ( X )] − φ ( m ) − s ( E [ X ] − m ) = 0 である。非負確率変数の期待値が零であるためD = 0 D=0 D = 0 X = m X=m X = m X = m X=m X = m
凹関数φ \varphi φ − φ -\varphi − φ ▨