In the past decade, seismologists have found that the topography in Colorado correlates with seismic velocities in the upper mantle, not with crustal thickness, and postulated that isostatic compensation of the southern Rocky Mountains takes place largely in the mantle. To test the validity of this hypothesis, we use Rayleigh wave phase and amplitude data recorded in the Rocky Mountain Front PASSCAL Experiment to obtain crustal thickness and shear-wave structure in the crust and upper mantle. The thickest crust in the model is beneath one of the two most elevated regions, the San Juan volcanic field. The other most elevated region in central Colorado, the Sawatch Range, is underlain by crust of anomalously low velocity. The seismically defined mantle lithosphere is thicker beneath the Great Plains than beneath the Colorado Plateau and is largely absent beneath the southern Rocky Mountains, but the correlation of topography with mantle velocities is weaker than the correlation with crustal anomalies. We construct a model that matches the observed Bouguer gravity anomaly primarily with variations in crustal thickness and density by assuming that density anomalies are proportional to the observed velocity anomalies. The anomalous mantle may also contribute to maintaining regional isostasy in Colorado, but its contribution may be much less important than the crust.