Consider a co-variance matrix A such that each element ai,j = E(Xi Xj) - E(Xi) E(Xj) where Xi,Xj are random variables. Consider the case that each variable X has a different sample size. Lets say that Xi contains the elements xi,1, …, xi,N, and Xj contains the elements xj,1, ..., xj,n where each element is paired up to element n and N > n. In this case, for each covariance ai,j, is it acceptable to trim the sample size for each Xi and Xj to n and continue the calculation? (I'm not sure if trim is the correct terminology but it seems to meet my needs). If it is acceptable to trim, then is it necessary to trim to the smallest n of all of the random variables X, or can I just trim to the smallest of the pair? I'd appreciate it if anyone can point me in the direction of some literature that explains this in detail. I've been struggling to find something that is specific to this case.