The infrared (IR) spectra of gem-quality xenotime crystals containing considerable amounts of rare earth elements (REEs), are characterized by sharp and strongly pleochroic absorption bands in the 3650–3350 cm−1 region. In contrast, the spectra of partially metamict samples are dominated by a broad band centered at around 3450 cm−1. Xenotime presents the interesting case of a nominally anhydrous mineral, where the OH stretching frequency region of weakly hydrogen-bonded OH groups is overlapped by absorption bands due to low-energetic f-f electron transitions of REEs, especially of dysprosium. In polarized spectra measured parallel to the c-axis, Dy shows a prominent sharp band at 3519 cm−1. The assignment of the REE bands is based on the polarized IR spectra of REE doped xenotime single crystals, which have been synthesized by the flux method. A single band at 3480 cm−1, strongly polarized perpendicular to the c-axis, is assigned to the stretching vibration of an OH group. Deuteration experiments performed at 950 °C prove the assignment of this band and the presence of additional structural OH groups, appearing at annealing temperatures above 500 °C. Models of the OH point defect incorporation into the crystal structure of xenotime can be derived on the basis of fully occupied cation sites and under the assumption of Y- and P-site vacancies. The water content of the gem-quality samples ranges from 5 to 10 wt ppm and for the partially metamict samples from 370 wt ppm to 1.7 wt% H2O.