Abstract:
We obtain the spectral decomposition of the hypergeometric differential operator on the contour $\operatorname{Re}z=1/2$. (The multiplicity of the spectrum of this operator is $2$.) As a result, we obtain a new integral transform different from the Jacobi (or Olevskii) transform. We also construct an ${}_3F_2$-orthogonal basis in a space of functions ranging in $\mathbb{C}^2$. The basis lies in the analytic continuation of continuous dual Hahn polynomials with respect to the index $n$ of a polynomial.