It is well known that artificially inducing large amplitude vibrations on buildings produces seismic waves that are detectable up to a few kilometers away. Does a similar effect occur with seismic tremors? If the tremor wave field were perturbed by the presence of buildings, passive surveys in a urban environment would be potentially impaired. The literature is rather inconclusive on this issue. We experimentally analyzed the cases of three of the most famous Italian towers: the leaning tower of Pisa, the bell tower of San Marco in Venice, and the Asinelli tower in Bologna. We also analyzed a large modern 16 story residential building. Even performing the measurements in windy days, we found no cases in which the large structures perturb the free-field tremor at distances larger than 12 m. This confirms what was expected from simple dimensional analysis and suggests that passive soil–structure interaction is of little concern for standard buildings and standard ambient conditions.