Mechanics of the continental lithosphere require the presence of a high-strength uppermost mantle that defines the “jelly sandwich” model for lithosphere strength layering. However, in deforming regions, growing numbers of geological and geophysical data predict a sub-Moho mantle strength lower than the crustal strength, or a “crème brûlée” model. To reconcile these two opposite views of lithosphere strength layering, we account for a new olivine rheology, which could promote some weakening during dynamic grain size reduction that enhances grain boundary sliding. We performed a one-dimensional numerical model of a deforming rock in order to quantify strain localization due to this weakening rheology. Strain localization occurs at temperatures lower than 800 °C and reaches a maximum at 610 °C, increasing the strain rate from 10−15 to >10−13 s−1. These results imply the existence of a sub-Moho ductile localizing mantle on a lithosphere scale, which occurs at Moho temperatures lower than 800 °C. Also, the localizing degree of this ductile mantle increases with decreasing Moho temperatures down to 550 °C. Such a ductile localizing mantle could therefore promote large strain localization during lithosphere deformation, like the brittle mantle that is commonly assumed in the “jelly sandwich” model. Furthermore, the long-term deformation (≥106 yr) of the ductile localizing mantle could change the lithosphere strength layering from “jelly sandwich” to “crème brûlée” in response to the grain boundary sliding–induced weakening in mantle shear zones.