The OH stretching vibration frequencies of crystalline zircon that contains water have been investigated by quantum mechanical calculation using CRYSTAL09 and several hybrid functionals. Incorporation mechanisms considered for H in zircon include: (1) hydrogarnet and partial hydrogarnet-type substitution; (2) H-compensated trivalent substitution at the Si site; and (3) H-compensated trivalent substitution at the Zr site. The results provide a clear picture of the H locations in zircon and their associated IR stretching vibrations. Based on the results of structural relaxation and corresponding OH stretching calculations, we can assign the 3420 cm−1 band to a hydrogarnet or partial hydrogarnet-type substitution with polarization dependent on the extent of local relaxation (i.e., closer to E||c for partial hydrogarnet and less polarized for hydrogarnet). These results also show that the 3385 cm−1 (E⊥c) IR band originates from H incorporation that accompanies a trivalent cation substitution at the Si-site. As an example of such a substitution, the [AlO4/H] defect was investigated. H-incorporation coupled with a trivalent cation substitution at the Zr-site appears to be responsible for generating the 3180 cm−1 (E⊥c) IR band. The ab initio calculation on [YO8/H] and [LaO8/H] structures show that the 3180 cm−1 band represents an average between two distinct bands at ~3220–3250 and ~3100 cm−1 attributed to H locations close to or further away from the Zr-substituted atom. The energetics of these substitutions demonstrate that the overlapping of the two IR bands depends on the substituting cation, and this is controlled, in part, by the thermal history of the sample and/or by the experimental conditions under which the IR spectra are obtained. Furthermore, these results predict for the first time the existence of a peroxy-type O bond associated with the partial hydrogarnet substitution in the zircon structure that now accounts for the ~800 cm−1 bands.