Mullite (Al4+2xSi2−2xO10−x) is known to exhibit a very distinct compositional variation in which the lattice constant in the a direction expands linearly, while that in the b direction slightly contracts with an increase of the x-value. In this study, first-principles density functional theory (DFT) simulations were applied to examine the cause. The atomic structure and charge density were examined. We found that the local charge redistribution due to a newly formed vacancy leads to the onset and recurrence of localized atomic relaxation. The charge redistribution and atomic relaxation causes the clockwise and counterclockwise rotations of the neighboring octahedral units. These rotations contribute to expansion in the a-axis and contraction in the b-axis. The mechanism is supported by the previous experimental measurements of the Al-O2 bond lengths projecting in the (001) plane. We conclude that the rotations of the octahedral units are the fundamental mechanism responsible for compositionally induced variations of mullite. Results derived from simulations also provide evidence for a preferred occurrence of oxygen vacancies parallel to the crystallographic b axis. It thus supports earlier findings of a partial ordering in mullite.