The Olympus Mons volcano on Mars is notable not only for its immense height and width, but also for substantial asymmetries in its structure. The gently sloped northwest flank extends to a much greater distance from the central caldera complex than the more steeply sloped southeast flank. Furthermore, the northwest flank exhibits lower-flank extensional faults, whereas the southeast shows upper-flank compressional terraces and lower-flank upthrust blocks. However, both the northwest and southeast flanks exhibit characteristic concave-upward profiles and steep bounding scarps, in contrast to other sectors. The NW-SE asymmetries are aligned with the regional slope from the Tharsis rise, but an understanding of the underlying causes has remained elusive. We use particle dynamics models of growing, spreading volcanoes to demonstrate that these flank structures could reflect the properties of the basement materials underlying Olympus Mons. We find that basal slopes alone are insufficient to produce the observed concave-upward slopes and asymmetries in flank extent and deformation style that are observed at Olympus Mons; instead, lateral variations in basal friction are required. These variations are most likely related to the presence of sediments, transported and preferentially accumulated downslope from the Tharsis rise. Such sediments likely correspond to ancient phyllosilicates (clays) recently discovered by the Mars Express mission.