The Closure Ordering of Nilpotent Orbits of the Complex Symmetric Pair $(\SO_{p+q},\SO_p\times\SO_q)$

http://dx.doi.org/10.4153/CJM-2003-046-3
Canad. J. Math. 55(2003), 1155-1190

  Published:2003-12-01
 Printed: Dec 2003

The main problem that is solved in this paper has the following simple formulation (which is not used in its solution). The group $K = \mathrm{O}_p ({\bf C}) \times \mathrm{O}_q ({\bf C})$ acts on the space $M_{p,q}$ of $p\times q$ complex matrices by $(a,b) \cdot x = axb^{-1}$, and so does its identity component $K^0 = \SO_p ({\bf C}) \times \SO_q ({\bf C})$. A $K$-orbit (or $K^0$-orbit) in $M_{p,q}$ is said to be nilpotent if its closure contains the zero matrix. The closure, $\overline{\mathcal{O}}$, of a nilpotent $K$-orbit (resp.\ $K^0$-orbit) ${\mathcal{O}}$ in $M_{p,q}$ is a union of ${\mathcal{O}}$ and some nilpotent $K$-orbits (resp.\ $K^0$-orbits) of smaller dimensions. The description of the closure of nilpotent $K$-orbits has been known for some time, but not so for the nilpotent $K^0$-orbits. A conjecture describing the closure of nilpotent $K^0$-orbits was proposed in \cite{DLS} and verified when $\min(p,q) \le 7$. In this paper we prove the conjecture. The proof is based on a study of two prehomogeneous vector spaces attached to $\mathcal{O}$ and determination of the basic relative invariants of these spaces. The above problem is equivalent to the problem of describing the closure of nilpotent orbits in the real Lie algebra $\mathfrak{so} (p,q)$ under the adjoint action of the identity component of the real orthogonal group $\mathrm{O}(p,q)$.