The acoustic approximation of elastic waves is a very common approximation in exploration geophysics. The interest of the acoustic approximation in the inverse problem context lies in the fact that it leads to a much lower numerical cost than for the elastic problem. Nevertheless, the earth is not an acoustic body, and it has been found in the past that this approximation is not without drawbacks mainly because of P-to-S energy conversion and that anisotropy cannot be easily modeled. We studied a different issue of this approximation related to small heterogeneities with respect to the minimum wavelength of the wavefield. We first numerically found that elastic and acoustic waves behave differently with respect to small-scale heterogeneities, introducing differences in amplitudes but also in phase between elastic and acoustic signals. We then derived physical and mathematical interpretations of this phenomenon, developing the different nature of elastic- and acoustic-wave propagation that led to the conclusion that, in rough media, acoustic waves can only be a poor-quality approximation of elastic waves. Interestingly, we also found that, in the acoustic case, small-scale heterogeneities give rise to natural acoustic effective anisotropic media through an anisotropic effective mass matrix. Unfortunately, this anisotropy is of a different nature compared with the elastic effective anisotropy and can not be used to mimic elastic anisotropy.