Abstract:
A Rayleigh triangle of size $n$ is a set of $n(n+1)/2$ real numbers $\lambda_{kl}$,
where $1\leqslant l\leqslant k\leqslant n$, which are
decreasing as $k$ increases for fixed $k$ and are increasing as $k$ increases for fixed $k-l$.
We construct a family of beta integrals over the space of Rayleigh triangles which interpolate matrix integrals of the types of Siegel, Hua Loo Keng, and Gindikin with respect to the dimension of the ground field
($\mathbb R$, $\mathbb C$, or the quaternions $\mathbb H$).
We also interpolate the Hua–Pickrell measures on the inverse limits of the symmetric spaces $\operatorname U(n)$,
$\operatorname U(n)/\operatorname O(n)$,
$\operatorname U(2n)/\operatorname{Sp}(n)$.
Our family of integrals also includes the Selberg integral.