Downstream grain-size fining in stratigraphy is driven primarily by selective deposition of sediment, and the long-term efficiency of this process is determined by: (1) the magnitude and characteristics of the input sediment supply; (2) the spatial distribution of subsidence rate, which creates accommodation for sediment preservation; and (3) the dynamics of sediment transport and deposition. A key challenge is to determine how these first two factors control the caliber and spatial distribution of deposits over time scales of 104–106 yr without incorporating sediment transport details that are largely unknowable for time-averaged stratigraphy in the geological past. We address this using grain-size data collected from fluvial conglomerates in the Eocene Pobla Basin, Spanish Pyrenees, a synorogenic basin where the timing of sediment deposition is well-constrained; the sediment budget is closed; and good exposure enables time lines within stratigraphy to be picked out unambiguously. For successive stratigraphic horizons, downstream trends in grain size and composition are derived for basin-filling sediment-routing systems with length scales of 6 and 40 km, respectively. Our data show that the rate of grain-size fining varies over time and with system length and can be linked to changes in source area. These results are contrasted with grain-size data from the Antist Group, a 60-km-long Oligocene system that mantles the Southern Pyrenees, where very slow rates of grain-size fining on the wedge top of this fold-and-thrust belt are observed. We apply a self-similarity–based selective deposition model to quantify the competing controls of tectonic subsidence and sediment supply on derived grain-size trends, and model results are compared with independent constraints on the Eocene–Oligocene evolution of the Pyrenees. Our results suggest that it is now possible to invert time-averaged grain-size trends in stratigraphy to gain quantitative information on the geological boundary conditions governing the evolution of sedimentary basins.