We use a novel numerical approach that fully couples the energy, momentum, and continuum equations to investigate the physics of extension and breakup of cold continental lithosphere to form new ocean basins. Unlike hot continental systems, where flat-lying detachment faults are nucleated in the strong part of the upper crust, cold continental systems have flat-lying detachment faults nucleating in the strong upper mantle at a relatively early stage. These detachment faults subsequently control the development of a mantle core complex and associated crustal structures. The observed structures are analogous to those developed in mid-crustal core complexes during extension of relatively thick and hot continental crust. In the cold environment, however, a strong elastic layer is developed within the mantle, shifting the stress-bearing part of the system to below the Moho. Our modeling results reproduce key tectonic elements of a natural system (the Iberia margin, offshore the Iberian Peninsula) by stretching a randomly perturbed, unpatterned lithosphere. Results also explain the “upper plate paradox” by doming of continental lithospheric mantle separated from the crust by two diffuse detachment zones dipping toward the two future continental margins. Doming is facilitated by channel flow of the lower crust.