In this study we conducted experiments with saturated columns packed with sand and glass beads to quantitatively examine surface roughness effect on deposition and release of micro- and nano-sized colloids at different solution ionic strengths. Experimental results showed more colloid retentions in both primary and secondary energy minima in sand than in glass bead columns, especially at high solution ionic strengths (e.g., >0.01 M). This observation cannot be explained by the classic Derjaguin–Landau–Verwey–Overbeek (DLVO) theory, which assumes sphere-smooth surface configuration. We modified the Derjaguin approximation approach and calculated interaction energies, which indicate that the sharp asperities on sand surfaces can facilitate colloid deposition in primary minima by reducing the energy barrier. In addition, the increased attachment in secondary minima in sand columns can be attributed to the presence of the valleys on sand surfaces where colloids associated at secondary minima can be shielded from hydrodynamic shear. Additional theoretical analysis verified that large valleys can locally increase the energy barrier as well as the secondary-minimum depth, and hence, are favorable for colloid deposition in secondary minima. Whereas the reduction effects of surface roughness on energy barrier has been extensively addressed in the literature, our modified DLVO analysis and experimental results demonstrate that the effect of this mechanism is only effective at high ionic strength for large colloids (e.g., >0.01 M for the 1156 nm colloid in this study). We provide experimental evidence and theoretical demonstration that surface roughness also plays an important role in colloid deposition at secondary minima under unfavorable conditions. Our study provides a more complete understanding of the effect of surface roughness on colloid deposition.