Olivine is a major rock-forming mineral in various magmatic and metamorphic rocks and the upper mantle. In this paper, we present the first high-precision analyses of olivine from 15 samples of magnesian skarns and silicate marbles (MSSM) from the collection of the Fersman Mineralogical Museum (Moscow, Russia). Mg# [Mg/(Mg+Fe2+)·100, mol%] of olivine from the samples studied varies from 86 to nearly 100. The main distinctive features of the olivine are anomalously low contents of Co (<51 mg/g), Cr (<5 mg/g), and Ni (<44 mg/g) and high content of B (23–856 mg/g), which correlate with host-rock compositions. Phosphorus (5–377 mg/g) incorporation in olivine is charge balanced by the incorporation of Li (0.15–61 mg/g) and Na (<14.3 mg/g). Y and REE contents exhibit positive correlations with Na, which suggests that REE incorporation into MSSM olivine could occur via charge-balanced coupled substitution with Na at low temperature and low aSiO2 conditions during MSSM formation. The documented compositional features of olivine from magnesian skarns and silicate marbles can help reconstruct the genesis of the host-rocks and identify xenocrysts of MSSM olivine in magmatic rocks.