Further properties of the linear sufficiency in the partitioned linear model

Chapter 1

Further properties of the linear sufficiency in the partitioned linear model

Augustyn Markiewicz and Simo Puntanen

AbstractA linear statisticFy, whereFis anf×nmatrix, is called linearly sufficient for estimable parametric functionKβ under the modelM ={y,Xβ,V}, if there exists a matrixAsuch thatAFyis the BLUE forKβ. In this paper we consider some particular aspects of the linear sufficiency in the partitioned linear model where X= (X₁:X₂)withβbeing partitioned accordingly. We provide new results and new insightful proofs for some known facts, using the properties of relevant covariance matrices and their expressions via certain orthogonal projectors. Particular attention will be paid to the situation under which adding new regressors (inX₂) does not affect the linear sufficiency ofFy.

Key words: Best linear unbiased estimator, generalized inverse, linear model, lin- ear sufficiency, orthogonal projector, L¨owner ordering, transformed linear model.

1.1 Introduction

In this paper we consider the partitioned linear modely=X₁β₁+X₂β₂+ε, or shortly denoted

M12={y,Xβ,V}={y,X₁β₁+X₂β₂,V}, (1.1) where we may drop off the subscripts fromM12if the partitioning is not essential in the context. In (1.1),yis ann-dimensional observable response variable, andε is an unobservable random error with a known covariance matrix cov(ε) =V=

Augustyn Markiewicz

Department of Mathematical and Statistical Methods, Pozna´n University of Life Sciences, Wojska Polskiego 28, PL-60637 Pozna´n, Poland, e-mail: amark@up.poznan.pl

Simo Puntanen

Faculty of Natural Sciences, FI-33014 University of Tampere, Finland, e-mail:

simo.puntanen@uta.fi

1

and Big Data : Selected Contributions from IWMS 2016. 2019, 1-22.

https://doi.org/10.1007/978-3-030-17519-1_1

cov(y)and expectation E(ε) =0. The matrixXis a knownn×pmatrix, i.e.,X∈ R^n×p, partitioned columnwise asX= (X₁:X₂),X_i∈R^n×pi,i=1,2.Vectorβ = (β⁰₁,β⁰₂)⁰∈R^pis a vector of fixed (but unknown) parameters; here symbol⁰stands for the transpose. Sometimes we will denoteµ=Xβ,µ_i=X_iβ_i,i=1,2.

As for notations, the symbols r(A), A⁻, A⁺,C(A), and C(A)^⊥, denote, re- spectively, the rank, a generalized inverse, the Moore–Penrose inverse, the col- umn space, and the orthogonal complement of the column space of the matrixA.

By A^⊥we denote any matrix satisfying C(A^⊥) =C(A)^⊥. Furthermore, we will writeP_A=P_C(A)=AA⁺=A(A⁰A)⁻A⁰ to denote the orthogonal projector (with respect to the standard inner product) onto C(A). In particular, we denote M= I_n−P_X,M_i=I_n−P_Xi,i=1,2.

In addition to the full model M12, we will consider the small models Mi= {y,X_iβ_i,V},i=1,2,and thereducedmodel

M12·2={M₂y,M₂X₁β₁,M₂VM₂}, (1.2) which is obtained by premultiplying the modelM12byM₂=I_n−P_X2. There is one further model that takes lot of our attention, it is thetransformedmodel

Mt={Fy,FXβ,FVF⁰}={Fy,FX₁β₁+FX₂β₂,FVF⁰}, (1.3) which is obtained be premultiplyingM12by matrixF∈R^f×n.

We assume that the models under consideration are consistent which in the case ofM means that the observed value of the response variable satisfies

y∈C(X:V) =C(X:VX^⊥) =C(X)⊕C(VX^⊥), (1.4) where “⊕” refers to the direct sum of column spaces.

Under the model M, the statistic Gy, whereGis an n×n matrix, is the best linear unbiased estimator, BLUE, of Xβ if Gyis unbiased, i.e., GX=X, and it has the smallest covariance matrix in the L¨owner sense among all unbiased linear estimators ofXβ; shortly denoted

cov(Gy)≤_Lcov(Cy) for allC∈R^n×n:CX=X. (1.5) The BLUE of an estimable parametric functionKβ, whereK∈R^k×p, is defined in the corresponding way. Recall thatKβ is said to be estimable if it has a linear unbiased estimator which happens if and only ifC(K⁰)⊂C(X⁰), i.e.,

Kβ is estimable underM ⇐⇒ C(K⁰)⊂C(X⁰). (1.6) The structure of our paper is as follows. In Section 1.2 we provide some prelim- inary results that are not only needed later on but they have some matrix-algebraic interest in themselves. In Sections 1.3 and 1.4 we consider the estimation ofµ=Xβ andµ₁=X₁β₁, respectively. In Section 1.5 we study the linear sufficiency under M1vs.M12. We characterize the linearly sufficient statisticFyby using the covari- ance matrices of the BLUEs underM12 and under its transformed versionMt. In

particular, certain orthogonal projectors appear useful in our considerations. From a different angle, the linear sufficiency in a partitioned linear model has been treated, e.g., in Isotalo & Puntanen (2006, 2009), Markiewicz & Puntanen (2009), and Kala

& Pordzik (2009). Baksalary (1984, 1987,§3.3,§5) considered linear sufficiency underM12andM1assuming thatV=I_n. Dong et al. (2014) study interesting con- nections between the BLUEs under two transformed models using so called matrix- rank method.

1.2 Some preliminary results

For the proof of the following fundamental lemma, see, e.g., Rao (1973, p. 282).

Lemma 1.Consider the general linear modelM ={y,Xβ,V}. Then the statistic Gyis theBLUEforXβ if and only ifGsatisfies the equation

G(X:VX^⊥) = (X:0), (1.7)

in which case we denoteG∈ {P_X|VX⊥}. The corresponding condition forByto be theBLUEof an estimable parametric functionKβ is

B(X:VX^⊥) = (K:0). (1.8) Two estimatorsG₁yandG₂yare said to be equal (with probability 1) whenever G₁y=G₂yfor ally∈C(X:V) =C(X:VX^⊥). When talking about the equality of estimators we sometimes may drop the phrase “with probability 1”. Thus for any G₁,G₂∈ {P_X|VX⊥}we haveG₁(X:VX^⊥) =G₂(X:VX^⊥), and therebyG₁y=G₂y with probability 1.

One well-known solution forGin (1.7) (which is always solvable) is

P_X;W⁻:=X(X⁰W⁻X)⁻X⁰W⁻, (1.9) whereWis a matrix belonging to the set of nonnegative definite matrices defined as

W =

W∈R^n×n:W=V+XUU⁰X⁰,C(W) =C(X:V) . (1.10) For clarity, we may use the notationW_A to indicate which model is under consid- eration. Similarly,W_A may denote a member of classW_A. We will also use the phrase “W_A is aW-matrix under the modelA”.

For the partitioned linear modelM12 we will say thatW∈W if the following properties hold:

W=V+XUU⁰X⁰=V+ (X₁:X₂) ^U1U

0

1 0

0 U₂U⁰₂

X⁰₁ X⁰₂

=V+X₁U₁U⁰₁X⁰₁+X₂U₂U⁰₂X⁰₂, (1.11a) W_i=V+X_iU_iU⁰_iX⁰_i, i=1,2, (1.11b) C(W) =C(X:V), C(W_i) =C(X_i:V), i=1,2. (1.11c) For example, the following statements concerningW∈W are equivalent:

C(X:V) =C(W), C(X)⊂C(W), C(X⁰W⁻X) =C(X⁰). (1.12) Instead ofW, several corresponding properties also hold in the extended set

W∗=

W∈R^n×n:W=V+XNX⁰, C(W) =C(X:V) , (1.13) whereN∈R^p×pcan be any (not necessarily nonnegative definite) matrix satisfying C(W) =C(X:V). However, in this paper we consider merely the setW. For further properties ofW∗, see, e.g., Puntanen et al. (2011,§12.3), and Kala et al. (2017).

Using (1.9), the BLUEs of µ=Xβ and of estimableKβ, respectively, can be expressed as

BLUE(Xβ |M) =µ(˜ M) =X(X⁰W⁻X)⁻X⁰W⁻y, (1.14a) BLUE(Kβ |M) =K(X⁰W⁻X)⁻X⁰W⁻y, (1.14b) whereWbelongs to the classW. The representations (1.14a)–(1.14b) are invariant with respect to the choice of generalized inverses involved; this can be shown using (1.12) and the fact that for any nonnullAandCthe following holds [Rao & Mitra (1971, Lemma 2.2.4)]:

AB⁻C=AB⁺Cfor allB⁻ ⇐⇒ C(C)⊂C(B)andC(A⁰)⊂C(B⁰). (1.15) Notice that partX(X⁰W⁻X)⁻X⁰ofP_X;W−in (1.9) is invariant with respect to the choice of generalized inverses involved but

P_X;W+=X(X⁰W⁺X)⁺X⁰W⁺=X(X⁰W⁻X)⁻X⁰W⁺ (1.16) for any choice ofW⁻and(X⁰W⁻X)⁻.

The concept of linear sufficiency was introduced by Baksalary & Kala (1981) and Drygas (1983) who considered linear statistics, which are “sufficient” forXβ underM, or in other words, “linear transformations preserving best linear unbiased estimators”. A linear statisticFy, whereF∈R^f×n, is called linearly sufficient for Xβ under the modelM if there exists a matrixA∈R^n×f such that AFy is the BLUE forXβ. Correspondingly,Fyis linearly sufficient for estimableKβ, where K∈R^k×p, if there exists a matrixA∈R^k×f such thatAFyis the BLUE forKβ.

Sometimes we will denote shortlyFy∈S(Xβ)orFy∈S(Xβ |M), to indicate thatFyis linearly sufficient forXβunder the modelM (if the model is not obvious from the context).

Drygas (1983) introduced the concept of linear minimal sufficiency and defined it as follows: Fyis linearly minimal sufficient if for any other linearly sufficient statisticsSy, there exists a matrixAsuch thatFy=ASyalmost surely.

In view of Lemma 1,Fyis linearly sufficient forXβif and only if the equation AF(X:VM) = (X:0) (1.17) has a solution forA. Baksalary & Kala (1981) and Drygas (1983) proved part (a) and Baksalary & Kala (1986) part (b) of the following:

Lemma 2.Consider the modelM ={y,Xβ,V}and letKβ be estimable. Then:

(a)The statisticFyis linearly sufficient forXβ if and only if

C(X)⊂C(WF⁰), whereW∈W. (1.18) Moreover, Fy is linearly minimal sufficient for Xβ if and only if C(X) = C(WF⁰).

(b)The statisticFyis linearly sufficient forKβ if and only if

C[X(X⁰W⁻X)⁻K⁰]⊂C(WF⁰), whereW∈W. (1.19) Moreover,Fyis linearly minimal sufficient forKβ if and only if equality holds in(1.19).

Actually, Kala et al. (2017) showed that in Lemma 2 the classW can be replaced with the more general classW∗defined in (1.13). For further related references, see Baksalary & Mathew (1986) and M¨uller (1987).

Supposing thatFyis linearly sufficient forXβ, one could expect that bothMand its transformed versionMt={Fy,FXβ,FVF⁰}provide the same basis for obtaining the BLUE ofXβ. This connection was proved by Baksalary & Kala (1981, 1986).

Moreover, Tian & Puntanen (2009, Th. 2.8) and Kala et al. (2017, Th. 2) showed the following:

Lemma 3.Consider the modelM ={y,Xβ,V}andMt={Fy,FXβ,FVF⁰},and letKβ be estimable underM12andMt. Then the following statements are equiva- lent:

(a)Fyis linearly sufficient forKβ.

(b) BLUE(Kβ|M) =BLUE(Kβ |Mt)with probability1.

(c)There exists at least one representation ofBLUEofKβ underM which is the BLUEalso under the transformed modelMt.

Later we will need the following Lemma 4. The proofs are parallel to those in Puntanen et al. (2011,§5.13), and Markiewicz & Puntanen (2015, Th. 5.2). In this

lemma the notationA^1/2stands for the nonnegative definite square root of a non- negative definite matrixA. SimilarlyA^+1/2denotes the Moore–Penrose inverse of A^1/2. Notice that in particularP_A=A^1/2A^+1/2=A^+1/2A^1/2.

Lemma 4.LetW,W₁andW₂be defined as in(1.11a)–(1.11c). Then:

(a)C(VM)^⊥=C(WM)^⊥=C(W⁺X:Q_W),whereQ_W=I_n−P_W, (b)C(W^1/2M)^⊥=C(W^+1/2X:Q_W),

(c)C(W^1/2M) =C(W^+1/2X:QW)^⊥=C(W^+1/2X)^⊥∩C(W), (d)P_W1/2M=P_W−P_W+1/2X=P_C(W)∩C(W+1/2X)^⊥.

Moreover, in(a)–(d)the matricesX,MandWcan be replaced withX_i,M_iandW_i, i=1,2, respectively, so that, for example,(a)becomes

(e)C(VM_i)^⊥=C(W_iM_i)^⊥=C(W⁺_i X_i:Q_Wi), i=1,2.

Similarly, reversing the roles ofXandM, the following, for example, holds:

(f)C(W⁺X)^⊥=C(WM:QW) and C(W⁺X) =C(VM)^⊥∩C(W).

Also the following lemma appears to be useful for our considerations.

Lemma 5.Consider the partitioned linear modelM12 and suppose thatF is an f×n matrix andW∈W.Then

(a)C(F⁰Q_FX2) =C(F⁰)∩C(M₂),whereQ_FX2=I_f−P_FX2, (b)C(WF⁰Q_FX2) =C(WF⁰)∩C(WM₂),

(c)C(W^1/2F⁰Q_FX2) =C(W^1/2F⁰)∩C(W^1/2M₂), (d)F⁰Q_FX2 =M₂F⁰Q_FX2.

Proof. In light of Rao & Mitra (1971, Complement 7, p. 118), we get

C(F⁰)∩C(M₂) =C[F⁰(FM^⊥₂)^⊥] =C(F⁰Q_FX2), (1.20) and so (a) is proved. In view of Lemma 4, we haveC(W^1/2M₂)^⊥=C(W^+1/2X₂: Q_W),and hence

C(W^1/2F⁰)∩C(W^1/2M₂) =C

W^1/2F⁰[FW^1/2(W^1/2M₂)^⊥]^⊥

=C

W^1/2F⁰[FW^1/2(W^+1/2X₂:Q_W)]^⊥

=C[W^1/2F⁰(FX₂)^⊥] =C(W^1/2F⁰Q_FX2). (1.21) Obviously in (1.21)W^1/2can be replaced withW. The statement (d) follows imme- diately from the inclusionC(F⁰Q_FX2)⊂C(M₂). ut

Next we present an important lemma characterizing the estimability underM12

andMt.

Lemma 6.Consider the modelsM12and its transformed versionMt and letFbe an f×n matrix. Then the followings statements hold:

(a)Xβ is estimable underMtif and only if

C(X⁰) =C(X⁰F⁰), i.e., C(X)∩C(F⁰)^⊥={0}. (1.22) (b)X₁β₁is estimable underM12if and only if

C(X⁰₁) =C(X⁰₁M₂), i.e., C(X₁)∩C(X₂) ={0}. (1.23) (c)X₁β₁is estimable underMt if and only if

C(X⁰₁) =C(X⁰₁F⁰Q_FX2), (1.24) or, equivalently, if and only if

C(X⁰₁) =C(X⁰₁F⁰) and C(FX₁)∩C(FX₂) ={0}. (1.25) (d)β is estimable underM12if and only ifr(X) =p.

(e)β₁is estimable underM12if and only ifr(X⁰₁M₂) =p₁.

(f)β₁is estimable underMt if and only ifr(X⁰₁F⁰Q_FX2) =r(X₁) =p1.

Proof. In view of (1.6),Xβ is estimable underMtif and only ifC(X⁰)⊂C(X⁰F⁰), i.e.,C(X⁰) =C(X⁰F⁰). The alternative claim in (a) follows from

r(FX) =r(X)−C(X)∩C(F⁰)^⊥, (1.26) where we have used the rank rule of Marsaglia & Styan (1974, Cor. 6.2) for the matrix product. For the claim (b), see, e.g., Puntanen et al. (2011,§16.1). To prove (c), we observe thatX₁β₁= (X₁:0)β is estimable underMt if and only if

C X⁰₁

0

⊂C X⁰₁F⁰ X⁰₂F⁰

, i.e.,X⁰₁=X⁰₁F⁰Aand0=X⁰₂F⁰A, (1.27) for someA. The equality0=X⁰₂F⁰Ameans thatA=Q_FX2Bfor someB, and thereby X⁰₁=X⁰₁F⁰Q_FX2Bwhich holds if and only ifC(X⁰₁) =C(X⁰₁F⁰Q_FX2).Thus we have proved condition (1.24). Notice that (1.24) is equivalent to

r(X⁰₁) =r(X⁰₁F⁰Q_FX2) =r(X⁰₁F⁰)−dimC(FX₁)∩C(FX₂)

=r(X₁)−dimC(X₁)∩C(F⁰)^⊥−dimC(FX₁)∩C(FX₂), (1.28) which confirms (1.25). The proofs of (d)–(f) are obvious. ut

For the proof Lemma 7, see, e.g., Puntanen et al. (2011, p. 152).

Lemma 7.The following three statements are equivalent:

P_A−P_Bis an orthogonal projector, P_A−P_B≥_L0, C(B)⊂C(A). (1.29) If any of the above conditions holds thenP_A−P_B=P_C(A)∩C(B)⊥=P_(I−PB)A.

1.3 Linearly sufficient statistic for µ = Xβ in M

Let us consider a partitioned linear model M12 ={y,X₁β₁+X₂β₂,V}, and its transformed versionMt ={Fy,FX₁β₁+FX₂β₂,FVF⁰}.ChoosingW=V+ XUU⁰X⁰∈W, we have, for example, the following representations for the covari- ance matrix of the BLUE forµ=Xβ:

cov(µ˜ |M12) =V−VM(MVM)⁻MV=W−WM(MWM)⁻MW−T

=W^1/2(I_n−P_W1/2M)W^1/2−T=W^1/2P_W+1/2XW^1/2−T

=X(X⁰W⁺X)⁻X⁰−T=X(X⁰W^+1/2W^+1/2X)⁻X⁰−T, (1.30) whereT=XUU⁰X⁰. Above we have used Lemma 4d which gives

I_n−P_W1/2M=Q_W+P_W+1/2X. (1.31) Consider then the transformed modelMtand assume thatXβis estimable under Mt, i.e., (1.22) holds. UnderMt we can choose theW-matrix as

W_Mt =FVF⁰+FXUU⁰X⁰F⁰=FWF⁰∈W_Mt, (1.32) and so, denotingT=XUU⁰X, we have

µ(˜ Mt) =BLUE(Xβ|Mt) =:G_ty

=X[X⁰F⁰(FWF⁰)⁻FX]⁻X⁰F⁰(FWF⁰)⁻Fy, (1.33) cov(µ˜|Mt) =X[X⁰F⁰(FWF⁰)⁻FX]⁻X⁰−T

=X(X⁰W^+1/2P_W1/2F⁰W^+1/2X)⁻X⁰−T. (1.34) Of course, by the definition of the BLUE, we always have the L¨owner ordering

cov(µ˜ |M12)≤_Lcov(µ˜ |Mt). (1.35) However, it is of interest to confirm (1.35) algebraically. To do this we see at once that

X⁰W^+1/2W^+1/2X≥LX⁰W^+1/2P_W1/2F⁰W^+1/2X. (1.36) Now (1.36) is equivalent to

(X⁰W^+1/2W^+1/2X)⁺≤_L(X⁰W^+1/2P_W1/2F⁰W^+1/2X)⁺. (1.37) Notice that the equivalence of (1.36) and (1.37) holds in view of the following result:

Let 0≤_LA≤_LB. Then A⁺≥_LB⁺ if and only if r(A) =r(B); see Milliken &

Akdeniz (1977). Now r(X⁰W⁺X) =r(X⁰W) =r(X),and

r(X⁰W^+1/2P_W1/2F⁰W^+1/2X) =r(X⁰W^+1/2P_W1/2F⁰)

=r(X⁰P_WF⁰) =r(X⁰F⁰) =r(X), (1.38) where the last equality follows from the estimability condition (1.25). Now (1.37) implies

X(X⁰W^+1/2W^+1/2X)⁻X⁰≤_LX(X⁰W^+1/2P_W1/2F⁰W^+1/2X)⁻X⁰, (1.39) which is just (1.35).

Now E(G_ty) =Xβ, and hence by Lemma 3,Fyis linearly sufficient forXβ if and only if

cov(µ˜ |M12) =cov(µ˜ |Mt). (1.40) Next we show directly that (1.40) is equivalent to (1.18). First we observe that (1.40) holds if and only if

X(X⁰W^+1/2W^+1/2X)⁻X⁰=X(X⁰W^+1/2P_W1/2F⁰W^+1/2X)⁻X⁰. (1.41) Pre- and postmultiplying (1.41) by X⁺ and by(X⁰)⁺, respectively, and using the fact thatP_X⁰ =X⁺X, gives an equivalent form to (1.41):

(X⁰W^+1/2W^+1/2X)⁺= (X⁰W^+1/2P_W1/2F⁰W^+1/2X)⁺. (1.42) Obviously (1.42) holds if and only ifC(W^+1/2X)⊂C(W^1/2F⁰),which further is equivalent to

C(X)⊂C(WF⁰), (1.43)

which is precisely the condition (1.18) forFybeing linearly sufficient forXβ. As a summary we can write the following:

Theorem 1.Letµ=Xβbe estimable underMtand letW∈W.Then

cov(µ˜ |M12)≤_Lcov(µ˜ |Mt). (1.44) Moreover, the following statements are equivalent:

(a) cov(µ˜ |M12) =cov(µ˜ |Mt),

(b)X(X⁰W⁺X)⁻X⁰=X(X⁰W^+1/2P_W1/2F⁰W^+1/2X)⁻X, (c)X⁰W⁺X=X⁰W^+1/2P_W1/2F⁰W^+1/2X,

(d)C(W^+1/2X)⊂C(W^1/2F⁰), (e)C(X)⊂C(WF⁰),

(f)Fyis linearly sufficient forµ=Xβ underM12.

1.4 Linearly sufficient statistic for µ

= X

β

in M

Consider then the estimation ofµ₁=X₁β₁underM12. We assume that (1.23) holds so thatµ₁is estimable underM12. Premultiplying the modelM12byM₂=I_n−P_X2 yields the reduced model

M12·2={M₂y, M₂X₁β₁, M₂VM₂}. (1.45) Now the well-known Frisch–Waugh–Lovell theorem, see, e.g., Groß & Puntanen (2000), states that the BLUEs ofµ₁underM12andM12·2coincide (with probability 1):

BLUE(µ₁|M12) =BLUE(µ₁|M12·2). (1.46) Hence, we immediately see thatM₂yis linearly sufficient forµ₁.

Now any matrix of the form

M₂VM₂+M₂X₁U₁U⁰₁X⁰₁M₂ (1.47) satisfying C(M₂V:M₂X₁U₁) =C(M₂V:M₂X₁),is a W-matrix inM12·2. We may denote this class asW_M12·2, and

W_M12·2=M₂WM₂=M₂W₁M₂∈W_M12·2, (1.48) whereWandW₁are defined as in (1.11a)–(1.11c).

It is interesting to observe that in (1.47) the matrixU₁can be chosen as a null matrix if and only if

C(M₂X₁)⊂C(M₂V), (1.49)

which can be shown to be equivalent to

C(X₁)⊂C(X₂:V). (1.50)

Namely, it is obvious that (1.50) implies (1.49) while the reverse implication follows from the following:

C(X₁)⊂C(X₁:X₂) =C(X₂:M₂X₁)⊂C(X₂:M₂V) =C(X₂:V). (1.51) This means that

M₂VM₂∈W_M12·2 ⇐⇒ C(X₁)⊂C(X₂:V). (1.52) One expression for the BLUE ofµ₁=X₁β₁, obtainable fromM12·2, is

BLUE(µ₁|M12) =µ˜₁(M12) =X₁(X⁰₁M˙_2WX₁)⁻X⁰₁M˙_2Wy, (1.53) where

M˙_2W =M₂W⁻_M

12·2M₂=M₂(M₂WM₂)⁻M₂. (1.54) In particular, if (1.50) holds then we can chooseW_M12·2=M₂VM₂,and

M˙_2W=M₂(M₂VM₂)⁻M₂=:M˙₂. (1.55) Notice that by Lemma 4d, we have

P_WM˙_2WP_W=P_WM₂(M₂WM₂)⁻M₂P_W

=W^+1/2P_W1/2M₂W^+1/2

=W^+1/2(P_W−P_W+1/2X₂)W^+1/2

=W⁺−W⁺X₂(X⁰₂W⁺X₂)⁻X⁰₂W⁺, (1.56) and hence, for example,

W ˙M_2WX₁=W[W⁺−W⁺X₂(X⁰₂W⁺X₂)⁻X⁰₂W⁺]X₁

= [I_n−X₂(X⁰₂W⁺X₂)⁻X⁰₂W⁺]X₁. (1.57) Observe that in (1.54), (1.56) and (1.57) the matrixWcan be replaced withW₁. For a thorough review of the properties ofM˙_2W, see Isotalo et al. (2008).

In the next theorem we collect some interesting properties of linearly sufficient estimators ofµ₁.

Theorem 2.Let µ₁=X₁β₁ be estimable under M12 and let W∈W. Then the statisticFyis linearly sufficient forµ₁underM12if and only if

C(W ˙M_2WX₁)⊂C(WF⁰), (1.58) or, equivalently,

C{[I_n−X₂(X⁰₂W⁺X₂)⁻X⁰₂W⁺]X₁} ⊂C(WF⁰), (1.59) whereM˙_2W =M₂(M₂WM₂)⁻M₂.Moreover,

(a)M₂yis linearly sufficient forµ₁.

(b)M˙_2Wy=M₂(M₂WM₂)⁻M₂yis linearly sufficient forµ₁. (c)X⁰₁M˙_2Wyis linearly minimal sufficient forµ₁.

(d)IfC(X₁)⊂C(X₂:V),(1.58)becomes

C(W ˙M₂X₁)⊂C(WF⁰), where M˙₂=M₂(M₂VM₂)⁻M₂. (1.60) (e)IfVis positive definite,(1.58)becomesC(M˙₂X₁)⊂C(F⁰).

(f)Ifβ₁is estimable underM12, then

Fy∈S(X₁β₁|M12) ⇐⇒ Fy∈S(β₁|M12). (1.61)

Proof. The sufficiency condition (1.58) was proved by Kala et al. (2017,§3), and, using a different approach, by Isotalo & Puntanen (2006, Th. 2). Claims (a), (b), (c) and (e) are straightforward to confirm and (d) was considered already before the Theorem. Let us confirm part (f). IfFy∈S(X₁β₁|M12), then there exists a matrix Asuch that

AF(X₁:X₂:VM) = (X₁:0:0). (1.62) Because of the estimability ofβ₁, the matrixX₁has a full column rank. Premulti- plying (1.62) by(X⁰₁X₁)⁻¹X⁰₁yields

BF(X₁:X₂:VM) = (I_p1:0:0), (1.63) whereB= (X⁰₁X₁)⁻¹X⁰₁A, and therebyFy∈S(X₁β₁|M12)impliesFy∈S(β₁| M12). The reverse direction can be proved in the corresponding way. Thus we have confirmed that claim (e) indeed holds. ut

The covariance matrix of the BLUE ofµ₁=X₁β₁underM12can be expressed as

cov(µ˜₁|M12) =X₁(X⁰₁M˙_2WX₁)⁻X⁰₁−T₁

=X₁[X⁰₁M₂(M₂WM₂)⁻M₂X₁]⁻X⁰₁−T₁

=X₁[X⁰₁W^+1/2P_W1/2M₂W^+1/2X₁]⁻X⁰₁−T₁, (1.64) whereT₁=X₁U₁U⁰₁X⁰₁andWcan be replaced withW₁.

Remark 1.The rank of the covariance matrix of the BLUE(β), as well as that of BLUE(Xβ), underM12is

r[cov(β˜|M12)] =dimC(X)∩C(V); (1.65) see, e.g., Puntanen et al. (2011, p. 137). Hence for estimableβ,

C(X)⊂C(V) ⇐⇒ cov(β˜|M12)is positive definite. (1.66) Similarly, for estimableβ₁,

r[cov(β˜₁|M12)] =r[cov(β˜₁|M12·2)] =dimC(M₂X₁)∩C(M₂VM₂)

=dimC(M₂X₁)∩C(M₂V)≤r(M₂X₁). (1.67) The estimability ofβ₁means that r(M₂X₁) =p₁and thereby

r[cov(β˜₁|M12)] =p₁ ⇐⇒ C(M₂X₁)⊂C(M₂V). (1.68) Thus, by the equivalence of (1.49) and (1.50), for estimableβ₁the following holds:

C(X₁)⊂C(X₂:V) ⇐⇒ cov(β˜₁|M12)is positive definite. ut (1.69)

What is the covariance matrix of the BLUE ofµ₁=X₁β₁underMt? First we need to make sure thatX₁β₁is estimable underMt, i.e., (1.24) holds.

Let us eliminate theFX₂β₂-part by premultiplying Mt byQ_FX2 =I_f−P_FX2. Thus we obtain the reduced transformed model

Mt·2={Q_FX2Fy,Q_FX2FX₁β₁,Q_FX2FVF⁰Q_FX2}

={N⁰y,N⁰X₁β₁,N⁰VN}, (1.70) whereN=F⁰Q_FX2∈R^n×f, and, see Lemma 5, the matrixNhas the property

C(N) =C(F⁰)∩C(M₂). (1.71) Notice also that in view of (1.71) and part (c) of Lemma 6,

r(N⁰X₁) =r(X₁)−dimC(X₁)∩C(N)^⊥

=r(X₁)−dimC(X₁)∩C[(F⁰)^⊥:X₂] =r(X₁), (1.72) so that

r(X⁰₁W^+1/2P_W1/2NW^+1/2X₁) =r(X⁰₁W^+1/2W^1/2N) =r(X⁰₁N) =r(X₁). (1.73) Correspondingly, we have

r(X⁰₁W^+1/2P_W1/2M₂W^+1/2X₁) =r(X₁). (1.74) TheW-matrix underMt·2can be chosen as

W_Mt·2=Q_FX2FW₁F⁰Q_FX2 =N⁰W₁N, (1.75) whereW₁can be replaced withW. In view of the Frisch–Waugh–Lowell theorem, the BLUE ofµ₁=X₁β₁is

µ˜₁(Mt) =µ˜₁(Mt·2) =X₁[X⁰₁N(N⁰WN)⁻N⁰X₁]⁻X⁰₁N(N⁰WN)⁻N⁰y, (1.76) while the corresponding covariance matrix is

cov(µ˜₁|Mt) =X₁[X⁰₁N(N⁰WN)⁻N⁰X₁]⁻X⁰₁−T₁

=X₁(X⁰₁W^+1/2P_W1/2NW^+1/2X₁)⁻X⁰₁−T₁, (1.77) whereT₁=X₁U₁U⁰₁X⁰₁(andWcan be replaced withW₁).

By definition we of course have

cov(µ˜₁|M12)≤_Lcov(µ˜₁|Mt), (1.78) but it is illustrative to confirm this also algebraically. First we observe that in view of Lemma 5,

C(W^1/2F⁰Q_FX2) =C(W^1/2F⁰)∩C(W^1/2M₂), (1.79) and thereby Lemma 7 implies thatP_W1/2M₂−P_W1/2F⁰Q_FX

2

=P_Z,where

C(Z) =C(W^1/2M₂)∩C(W^+1/2F⁰Q_FX2)^⊥. (1.80) Hence we have the following equivalent inequalities:

X⁰₁W^+1/2(P_W1/2M2−P_W1/2F⁰QFX2

)W^+1/2X₁≥_L0, (1.81) X⁰₁W^+1/2P_W1/2M2W^+1/2X₁≥_LX⁰₁W^+1/2P_W1/2F⁰QFX2

W^+1/2X₁, (1.82) (X⁰₁W^+1/2P_W1/2M2W^+1/2X₁)⁺≤_L(X⁰₁W^+1/2P_W1/2F⁰QFX2

W^+1/2X₁)⁺. (1.83) The equivalence between (1.82) and (1.83) is due to the fact that the matrices on each side of (1.82) have the same rank, which is r(X₁); see (1.73) and (1.74). The equivalence between (1.83) and (1.78) follows by the same argument as that be- tween (1.41) and (1.42).

The equality in (1.82) holds if and only if P_ZW^+1/2X₁= (P_W1/2M₂−P_W1/2F⁰Q_FX

2

)W^+1/2X₁=0, (1.84) which is equivalent to

W^1/2(P_W1/2M2−P_W1/2F⁰QFX2

)W^+1/2X₁=0. (1.85) Writing up (1.85) yields

W ˙M_2WX₁=WM₂(M₂WM₂)⁻M₂X₁=WN(N⁰WN)⁻NX₁. (1.86) We observe that in view of (1.80) we have

C(Z)^⊥=C(W^+1/2X₂:Q_W:W^1/2F⁰Q_FX2), (1.87) where we have used the Lemma 4 giving usC(W^1/2M₂)^⊥=C(W^+1/2X₂:Q_W).

Therefore (1.84) holds if and only if

C(W^+1/2X₁)⊂C(Z)^⊥=C(W^+1/2X₂:Q_W:W^1/2F⁰Q_FX2). (1.88) Premultiplying the above inclusion byW^1/2yields an equivalent condition:

C(X₁)⊂C(X₂:WF⁰Q_FX2) =C(X₂:M₂WF⁰Q_FX2)

=C(X₂)⊕[C(WF⁰)∩C(WM₂)]. (1.89) Our next step is to prove the equivalence of (1.89) and the linear sufficiency condition

C(W ˙M_2WX₁)⊂C(WF⁰). (1.90) The equality (1.85), which is equivalent to (1.89), immediately implies (1.90). To go the other way, we observe that (1.90) implies

C[WM₂(M₂WM₂)⁻M₂X₁]⊂C(WF⁰)∩C(WM₂) =C(WF⁰Q_FX2), (1.91) where we have used Lemma 5. Premultiplying (1.91) byM₂and noting that

M₂WM₂(M₂WM₂)⁺=P_M2W (1.92) yields

C(P_M2WM₂X₁) =C(M₂X₁)⊂C(M₂WF⁰Q_FX2). (1.93) Using (1.93) we get

C(X₁)⊂C(X₁:X₂) =C(X₂:M₂X₁)⊂C(X₂:M₂WF⁰Q_FX2), (1.94) and thus we have shown that (1.90) implies (1.89).

Now we can summarise our findings for further equivalent conditions for Fy being linearly sufficient forX₁β₁:

Theorem 3.Letµ₁=X₁β₁be estimable underM12andMtand letW∈W.Then cov(µ˜₁|M12)≤_Lcov(µ˜₁|Mt). (1.95) Moreover, the following statements are equivalent:

(a) cov(µ˜₁|M12) =cov(µ˜₁|Mt).

(b)C(W ˙M_2WX₁)⊂C(WF⁰).

(c)C(X₁)⊂C(X₂:WF⁰Q_FX2) =C(X₂:M₂WF⁰Q_FX2).

(d)C(X₁)⊂C(X₂)⊕[C(WF⁰)∩C(WM₂)].

(e)WM₂(M₂WM₂)⁻M₂X₁=WN(N⁰WN)⁻NX₁, whereN=F⁰Q_FX2. (f)The statisticFyis linearly sufficient forX₁β₁underM12.

If, in the situation of Theorem 3, we requestFyto be linearly sufficient forX₁β₁ foranyX₁(expecting thoughX₁β₁to be estimable), we get the following corollary.

Corollary 1.Letµ₁=X₁β₁be estimable underM12andMtand letW∈W.Then the following statements are equivalent:

(a)The statisticFyis linearly sufficient forX₁β₁underM12for anyX₁. (b)C(W)⊂C(X₂:WF⁰Q_FX2) =C(X₂)⊕C(WF⁰)∩C(WM₂).

Proof. The statisticFyis linearly sufficient forX₁β₁underM12for anyX₁if and only if

W^+1/2P_ZW^+1/2=0. (1.96)

Now (1.96) holds if and only if

C(W^+1/2)⊂C(Z)^⊥=C(W^+1/2X₂:Q_W:W^1/2F⁰Q_FX2). (1.97) Premultipying (1.97) byW^1/2yields an equivalent form

C(W)⊂C(X₂:WF⁰Q_FX2) =C(X₂)⊕C(WF⁰)∩C(WM₂). ut (1.98)

1.5 Linear sufficiency under M

vs. M

Consider the small modelM1={y,X₁β₁,V}and full modelM12={y,X₁β₁+ X₂β₂,V}. Here is a reasonable question: what about comparing conditions for

Fy∈S(µ₁|M1) versus Fy∈S(µ₁|M12). (1.99) For example, under which condition

Fy∈S(µ₁|M1) =⇒ Fy∈S(µ₁|M12). (1.100) There is one crucial matter requiring our attention. Namely in the small modelM1

the response y is lying inC(W₁)but in M12 the response y can be in a wider subspaceC(W). How to take this into account? What about assuming that

C(X₂)⊂C(X₁:V)? (1.101)

This assumption means that adding theX₂-part into the model does not carryyout ofC(W₁)which seems to be a logical requirement. In such a situation we should find conditions under which

C(X₁)⊂C(W₁F⁰) (1.102)

implies

C(W₁M˙_2WX₁)⊂C(W₁F⁰). (1.103) We know that under certain conditions the BLUE ofX₁β₁does not change when the predictors inX₂are added into the model. It seems obvious that in such a sit- uation (1.102) and (1.103) are equivalent. Supposing that (1.101) holds, then, e.g., according to Haslett & Puntanen (2010, Th. 3.1),

µ˜₁(M12) =µ˜₁(M1)−X₁(X⁰₁W⁺₁X₁)⁻X⁰₁W⁺₁µ˜₂(M12), (1.104) and hence

µ˜₁(M12) =µ˜₁(M1) (1.105) if and only ifX⁰₁W⁺₁µ˜₂(M12) =0,i.e.,

X⁰₁W⁺₁X₂(X⁰₂M˙₁X₂)⁻X⁰₂M˙₁y=0. (1.106)

Requesting (1.106) to hold for ally∈C(X₁:V)and using the assumptionC(X₂)⊂ C(X₁:V), we obtain

X⁰₁W⁺₁X₂(X⁰₂M˙₁X₂)⁻X⁰₂M˙₁X₂=0, (1.107) i.e.,

X⁰₁W⁺₁X₂P_X⁰

2=X⁰₁W⁺₁X₂=0, (1.108) where we have used the factC(X⁰₂M˙₁X₂) =C(X⁰₂). Thus we have shown the equiv- alence of (1.105) and (1.108).

On the other hand, (1.102) implies (1.103) if and only if C(W₁M˙_2WX₁)⊂ C(X₁),which is equivalent to

C(W₁M˙_2WX₁) =C(X₁), (1.109) because we know that r(W₁M˙_2WX₁) =r(X₁). Hence neither column spaces in (1.109) can be a proper subspace of the other. Therefore, as stated by Baksalary (1984, p. 23) in the case ofV=I_n, either the classes of statistics which are linearly sufficient forµ₁are in the modelsM1andM12 exactly the same, or, if not, there exists at least one statisticFysuch thatFy∈S(µ₁|M1)butFy∈/S(µ₁|M12)and at least one statisticFysuch thatFy∈S(µ₁|M12)butFy∈/S(µ₁|M1).

Now (1.102) and (1.103) are equivalent if and only if (1.109) holds, i.e.,

M₁W₁M˙_2WX₁=0. (1.110)

Using (1.57), (1.110) becomes

M₁X₂(X⁰₂W⁺₁X₂)⁻X⁰₂W⁺₁X₁=0. (1.111) Because r(M₁X₂) =r(X₂), we can cancel, on account of Marsaglia & Styan (1974, Th. 2), the matrixM₁in (1.111) and thus obtain

X₂(X⁰₂W⁺₁X₂)⁻X⁰₂W⁺₁X₁=0. (1.112) Premultiplying (1.111) byX⁰₂W⁺₁ shows that (1.109) is equivalent to

X⁰₂W⁺₁X₁=0. (1.113)

In (1.113) of courseW⁺₁ can be replaced with anyW⁻₁. Thus we have proved the following:

Theorem 4.Consider the models M12 and M1 and suppose that µ₁=X₁β₁ is estimable underM12 andC(X₂)⊂C(X₁:V). Then the following statements are equivalent:

(a)X⁰₁W⁺₁X₂=0,

(b) BLUE(µ₁|M1) =BLUE(µ₁|M12)with probability1, (c)Fy∈S(µ₁|M1) ⇐⇒ Fy∈S(µ₁|M12).

Overlooking the problem forybelonging toC(X:V)or toC(X₁:V), we can start our considerations by assuming that (1.100) holds, i.e.,

C(X₁)⊂C(W₁F⁰) =⇒ C(W ˙M_2WX₁)⊂C(WF⁰). (1.114) ChoosingF⁰=W⁻₁X₁we observe thatFy∈S(µ₁|M1)for any choice ofW⁻₁. Thus (1.114) implies that we must also have

C(W ˙M_2WX₁)⊂C(WW⁻₁X₁). (1.115) According to Lemma 3 of Baksalary & Mathew (1986), (for nonnullW ˙M_2WX₁and X₁) the inclusion (1.115) holds for anyW⁻₁ if and only if

C(W)⊂C(W₁) (1.116)

holds along with

C(W ˙M_2WX₁)⊂C(WW⁺₁X₁). (1.117) Inclusion (1.116) means thatC(X₂)⊂C(X₁:V), i.e.,C(W) =C(W₁), which is our assumption in Theorem 4. Thus we can also conclude the following.

Corollary 2.Consider the modelsM12 andM1and suppose that µ₁=X₁β₁ is estimable underM12. Then the following statements are equivalent:

(a)X⁰₁W⁺₁X₂=0andC(X₂)⊂C(X₁:V).

(b)Fy∈S(µ₁|M1) ⇐⇒ Fy∈S(µ₁|M12).

We complete this section by considering the linear sufficiency ofFyversus that ofFM₂y.

Theorem 5.Consider the modelsM12 andM12·2and suppose thatµ₁=X₁β₁is estimable underM12. Then

(a)Fy∈S(µ₁|M12) =⇒ FM₂y∈S(µ₁|M12).

(b)The reverse relation in(a)holds ⇐⇒ C(FM₂W)∩C(FX₂) ={0}.

Moreover, the following statements are equivalent:

(c)FM₂y∈S(X₁β₁|M12), (d)FM₂y∈S(X₁β₁|M12·2), (e)FM₂y∈S(M₂X₁β₁|M12·2).

Proof. To prove (a), we observe thatFy∈S(µ₁|M12)implies (1.91), i.e.,

C(W ˙M_2WX₁)⊂C(WF⁰Q_FX2). (1.118) Now on account Lemma 5d, we haveM₂F⁰Q_FX2 =F⁰Q_FX2. Substituting this into (1.118) givesC(W ˙M_2WX₁)⊂C(WM₂F⁰Q_FX2), and so

C(W ˙M_2WX₁)⊂C(WM₂F⁰), (1.119)