Rare-earth silicate oxy-apatites have been found to exhibit high ion-conductivity along channels within their structure, which makes them candidate materials for solid oxide fuel cell electrolytes. It is not understood so far why this high ion-conductivity is restricted to oxide-ion transport and does not occur for halide ions common in apatite type minerals. This study reports on the relationship between the topology of these structural channels and the spatial displacement of the chloride ion in three different structures of natural apatite group minerals [Pb5(BO4)3Cl (Z = 2) with B = P (pyromorphite), V (vanadinite), and As (mimetite)] using single-crystal X-ray diffraction. All of these minerals crystallize in the hexagonal chlorapatite structure with space group P63/m with no symmetry lowering or site splitting. The anion channel is built from a face-sharing array of nearly regular Pb26 octahedra running parallel to the c-axis, and the chloride ions were found at the center of each octahedron with bond-valence sums of 1.10 for mimetite and vanadinite and 1.25 for pyromorphite. The mean square displacement (msd) of the chloride ion in  was found to be a function of the size of Pb26 octahedron in that direction. This position is also the center of a flat O36 trigonal antiprism. The msds of the chloride ion in the x-y plane were found to be correlated to the size of the antiprism in this plane, namely the distance between the chloride ion and its nearest oxide ions, and the amount of roto-oscillation motion of the BO4 tetrahedra around the B-O1 axis. While the chloride ion in the channel was bonded to six Pb cations, the repulsion from six neighboring oxygen ions is also apparent and this repulsion restricts the motion of the chloride ion within the x-y plane.