Jahn-Teller driven quadrupolar ordering and spin-orbital dimer formation in GaNb4Se8

The lacunar spinel GaNb4Se8 is a tetrahedral cluster Mott insulator where spin-orbit coupling on molecular orbitals and Jahn-Teller energy scales are competitive. GaNb4Se8 undergoes a structural and antipolar ordering transition at TQ=50 K that corresponds to a quadrupolar ordering of molecular orbitals on Nb4 clusters. A second transition occurs at TM=29 K, where local distortions on the Nb4 clusters rearrange. We present a single crystal x-ray diffraction investigation for these phase transitions and solve the crystal structure in the intermediate TM<T<TQ and low T<TM temperature phases. The intermediate phase is a primitive cubic P213 structure with a staggered arrangement of Nb4 cluster distortions. A symmetry mode analysis reveals that the transition at TQ is continuous and described by a single Jahn-Teller active distortion mode. In the low temperature phase, the symmetry of Nb4 clusters is further reduced and a staggered arrangement of intercluster dimers is formed, suggesting a valence bond solid magnetic state.