On the background of Early Paleozoic precollisional, syncollisional, and late collisional igneous rocks prevailing in the region (Khaidai and Shara-Nur granitoids and Birkhin gabbroid complex), rare-metal pegmatoid granite bodies and pegmatites along the southern edge of the Ol’khon region are of particular interest. They have a Middle Paleozoic age (390–391 Ma), cut the Caledonides, and belong to different geochemical types. The Anga unit includes amazonite-containing Li–F–B pegmatites, which are also enriched in Ta, Nb, and W. In the Ol’khon Group, only one K-feldspathic body rich in Rb, Be, Nb, W, Sn, Sc, U, and Th, with large crystals of aquamarine, has been found. The compositions of granite–pegmatite bodies and accessory rare-metal minerals have been studied. The rare-metal granite–pegmatites probably form a peripheral zone of the Hercynian within-plate setting widespread in the eastern Baikal area and related to the influence of the Siberian hot spot.