When a free‐standing column with a given base becomes taller and taller, there is a competition between the increase in its size (more stable) and the increase in its slenderness (less stable). This article investigates how these two competing phenomena affect the stability of tall, slender, free‐standing columns when subjected to horizontal and vertical ground shaking. The main conclusion of the article is that the outcome of this competition is sensitive to local details of the ground shaking and the dominant frequency of a possible coherent, distinguishable pulse. The often‐observed increase in stability due to increase in height (despite the increase in slenderness) may be further enhanced due to a sudden transition from the lower mode of overturning with impact to the higher mode of overturning without impact. The article proceeds by offering a simple mathematical explanation why the vertical ground acceleration has a marginal effect on the stability of a slender, free‐standing column and concludes that the level of ground shaking that is needed to overturn a tall free‐standing column of any size and any slenderness is a decreasing function of the length scale of the dominant coherent acceleration pulse normalized to the base width of the column.