Caesium-bearing phosphates with the β-tridymite structure are considered as storage container for radioactive Cs. In this context, double phosphates of caesium and divalent cations, CsM2+PO4 where M2+ = Zn, Co, Mn, Mg, and Ni, have been synthesised by the precipitation method as powders and investigated by infrared spectroscopy in combination with factor-group analyses for band assignment. Comparison of the IR pattern of CsZnPO4 and CsCoPO4 confirms that they are isomorphic (space group P1 21/a 1 at room conditions). Both undergo T-induced phase transitions: one at 260 °C (Pn21a) and another one at 310 °C (Pnma). High-temperature IR spectra of the orthorhombic modifications of CsZnPO4 are presented and assigned. Comparison of the IR-spectrum of CsMnPO4 with that of CsZnPO4 (Pn21a) confirms that it crystallizes in the orthorhombic structure at room conditions (Pn21a). CsMgPO4 crystallizes at room conditions in the orthorhombic space group Pnma, however, it tends to hydrate immediately after quenching in air forming the cubic hydrate CsMgPO4·6H2O (space group F4̄3m). The liability to hydration is also characteristic for CsNiPO4. Temperature-dependent IR spectroscopy, X-ray diffraction, thermal analyses (TGA, DTA) and second-harmonic generation (SHG) revealed that pure anhydrous CsMgPO4 (space group Pnma) can only be stored in air at temperatures higher than 150 °C. Our study proves that CsM2+PO4, where M2+ = Zn, Co, Mn, may be stable containers for radioactive Cs at room temperatures. However, CsMgPO4 and CsNiPO4 can only be used as crystalline matrices for the immobilisation of radioactive Cs when stored at temperature higher than 150 °C due to their hydrolytic instability in air. The difference in the structures of CsMg phosphate and Cs phosphates with Zn, Co, and Mn is caused by the different liability of these cations to form either ionic or covalent chemical bonds.