Canted magnetism, spin interactions, and anisotropic magnetodielectric response in CoTeMoO₆

We investigate the chiral magnet CoTeMoO₆ (CTMO), an orthorhombic P21212 system with a canted antiferromagnetic structure and weak ferromagnetism along the a axis. Despite lacking a unique polar axis in its crystal structure, CTMO exhibits significant magnetodielectric coupling, which is consistent with a spin-dependent p-d hybridization mechanism. Using single-crystal magnetization, neutron diffraction, and magnetocapacitance measurements, complemented by inelastic neutron scattering, we identify that the magnetic behavior of CTMO can be described by an anisotropic Hamiltonian with magnetic moments interpolating between two limit representations: spin-orbit entangled Jeff=1/2 and spin-only S=3/2 moments. Depending on the representation, a Dzyaloshinskii-Moriya interaction or a site-dependent off-diagonal single-ion anisotropy leads to the formation of a canted magnetic structure. A comparative study with MnTeMoO₆ (MTMO), which has a collinear antiferromagnetic structure and negligible magnetodielectric coupling, confirms the crucial role of noncollinearity and intrinsic magnetic interactions in spin-driven dielectric responses. These results highlight the broader relevance of the p-d hybridization mechanism in insulating magnets and motivate further studies on magnetoelectric coupling in CTMO.