Quantifying how rock properties influence landscape evolution is an outstanding challenge that confounds attempts to generalize relationships between topography and erosion rate. Defining proxies for rock strength at the landscape scale is difficult because hillslope and channel erosion processes differ in their sensitivity to rock material properties. Here we show how the fracture density of bedrock hillslopes and the related grain size distribution of sediment delivered to channels exert a first-order control on topography and erosion rate in steep, rocky landscapes. We analyzed two sites in southern California, USA, that share similar lithology and hydroclimate, but that differ in tectonic setting, bedrock fracture density, sediment grain size, and relief structure. Multi-scale analysis of lidar point clouds reveals a systematic contrast in mean slope and relief of bedrock cliffs, consistent with higher fracture density leading to a reduction in rock mass strength, but colluvial slopes are invariant across both landscapes. The relation between fluvial relief and erosion rate indirectly records differences in fracture density via the grain-size distribution observed in channels. By increasing the fluvial incision threshold, coarser sediment decreases the frequency of erosive floods. Erosion thresholds also appear to control the extent of headwater colluvial channel networks likely carved by debris flows. Taken together, these results show how increased bedrock fracturing alters drainage density and systematically reduces landscape-scale topographic relief.