The USArray ground‐motion visualization (GMV) is an Incorporated Research Institutions for Seismology (IRIS) video product that illustrates how seismic waves travel away from an earthquake by depicting seismometers as symbols that vary in color according to the recorded amplitudes. GMVs are typically the most popular product the IRIS produces following an earthquake (e.g., unique views for a recent Oklahoma earthquake). Many instructors feel that dynamic visualizations offer learning advantages over static media when demonstrating dynamic processes, but research indicated they can impede learning by placing greater information processing requirements on the learner. We sought to evaluate changes in student understanding of seismic waves from GMVs by collecting data from three different college‐level settings: general student population in a psychology laboratory (novices), students in middle‐ and upper‐level geoscience courses (geoscience majors), and a seismology research group. A seven‐question multiple‐choice assessment was developed for use in all three settings and then administered in the laboratory and classroom. Using a similar question before and after the GMV viewing, we found that most geoscience majors understood seismic‐wave concepts prior to the GMV and the GMV improved their understanding. Only about half of the novices appeared to understand seismic‐wave concepts prior to the GMV and performance decreased after the GMV. Performance decreases were larger when students watched an alternative tutorial GMV developed to further illustrate what a GMV represents. An increase in the breadth of incorrect answer selections by novices indicates they became more confused about what happens to energy from an earthquake when shown a GMV. Lower performance on other post‐GMV questions by novices suggests that the current style of GMVs are unable to teach basic seismological concepts to people who do not have some formal geoscience training. Although web traffic to GMVs indicates people’s interest in watching the videos, watching GMVs does not appear to translate to improved understanding of seismic waves for novices. Future development of dynamic visualizations such as GMVs should consider the cognitive load these learning materials impose on the learner and seek to further implement principles of multimedia instructional design that minimize cognitive processing demands.