The presence of water (hydrogen) in nominally anhydrous mantle minerals may have profound effects on their physical properties (e.g., electrical conductivity, diffusivity, rheology), and these effects are expected to depend on how water is incorporated in the crystal structure. For olivine, the most abundant upper mantle mineral, despite extensive studies, mostly using vibrational spectroscopy, the interpretations are still not well constrained. To provide better understanding on this issue, we carried out a comprehensive 1H and 29Si NMR study on an Mg2SiO4 forsterite sample containing about 0.5 wt% H2O synthesized at 12 GPa and 1200 °C, complemented by Raman measurement and first-principles calculation of the geometry, stability, and NMR parameters of model structures. The Raman spectra contain relatively sharp O-H stretching bands near 3612, 3579, and 3567 cm−1 and a broader band near 3547 cm−1, similar to previous reports. The 1H static and MAS NMR data revealed that there are two main populations of protons in the hydrous forsterite structure, one experiencing strong 1H-1H homonuclear dipolar couplings and contributing to a broad peak near 2.4 ppm, and another with weaker dipolar couplings and contributing to a narrower peak near 1.2 ppm in the MAS NMR spectrum at 30 kHz. Two-dimensional 1H CRAMPS-MAS NMR measurements confirmed that the two proton components belong to the same phase and the contrast in MAS NMR peak width is largely due to difference in the strength of 1H-1H homonuclear dipolar couplings. In addition, there is also a very weak, narrow 1H MAS NMR peak near 7.3 ppm (contributing to <0.1% of the total intensity) due to protons that are more remote from the two main components. First-principles calculation confirmed that the two main proton components can be attributed to the hydrogarnet-like substitution mechanism of four H ions for one Si [(4H)Si] in a tetrahedral site of olivine, but unlike hydrogarnet with one of the protons pointing away from the tetrahedral center and located in an adjacent interstitial site, thus experiencing weaker dipolar couplings than those in the vicinity of the vacant tetrahedron; the very weak narrow peak near 7.3 ppm can be attributed to the substitution mechanism of two H ions for one Mg in an M1 site [(2H)M1] of forsterite. The 1H-29Si CP-MAS NMR spectra revealed both a broad peak encompassing the position for OH defect-free forsterite (−61.7 ppm) and a narrower peak at higher frequency (−60.9 ppm). First-principles calculation indicates that these peaks are accountable by the same models as for the 1H NMR data. Thus, this study has provided unambiguous evidence supporting that hydrogen is incorporated in forsterite at relatively high-pressure dominantly as (4H)Si defects, with (2H)M1 defects playing only a very minor role. The much larger 1H chemical shift for protons associated with the latter (than the former) is correlated with stronger hydrogen bonding for the latter, which in turn reflects difference in bonding environments of the OH groups (with the latter bonded to a Si, and the former only bonded to Mg). Similar correlation applies to the O-H stretching frequency. The (4H)Si defects are responsible for the observed high-frequency O-H stretching bands (>3450 cm−1), and the (2H)M1 defects give lower frequencies (undetected here due to low abundance, but most likely near 3160–3220 cm−1 as previously reported) in vibrational spectra. These results can serve as a guide for (re-)interpretation of infrared and Raman spectroscopic data on hydrous olivine produced under different pressure and silica activity conditions, and require reconsideration of any models for the effects of water on physical properties of olivine based on different interpretations of such data. This study also demonstrated the usefulness of the combined solid-state NMR and first-principles calculation approach in unraveling the hydrogen incorporation mechanisms in nominally anhydrous minerals.