Tropical glaciers have retreated alongside warming temperatures over the past century, yet the way in which these trends fit into a long-term geological context is largely unclear. Here, we present reconstructions of Holocene glacier extents relative to today from the Quelccaya ice cap (Peru) and the Rwenzori Mountains (Uganda) based on measurements of in situ14C and 10Be from recently exposed bedrock. Ice-extent histories are similar at both sites and suggest that ice was generally smaller than today during the first half of the Holocene and larger than today for most, if not all, of the past several millennia. The similar glaciation history in South America and Africa suggests that large-scale warming followed by cooling of the tropics during the late Holocene primarily drove ice extent, rather than regional changes in precipitation. Our results also imply that recent tropical ice retreat is anomalous in a multimillennial context.