Asymptotic Behavior of the Error Probabilities of the First and Second Kind in Hypothesis Testing of the Spectrum of a Stationary Gaussian Process

This paper derives expressions for the rate of convergence to zero of errors of the first and second kind, $\alpha_n$ and $\beta_n$ respectively, according to the Neyman–Pearson criterion for different hypotheses $H_i$, $i=1,2,$ and for known sampling $x_k=x(Tk/n)$, $k=0,1,\dots,n$, for $n\to\infty$. Hypothesis $H_i$ states that the spectral density of the process $x(t)$ is equal to $f_i(\lambda)\asymp|\lambda|^{(-1+\lambda_i)}$ for$\lambda\to\infty$. In order to derive the asymptotic behavior of $\lambda_n$ and $\beta_n$ the distribution of eigenvalues of the matrix $B_1B_2^{-1}$ is determined, where $B_1$ is the correlation matrix of the random variables $x_k$, $k=0,1,\dots,n$, under the condition that hypothesis $H_i$ holds. Afterwards the theorem on large deviations for independent random variables is applied.