Seismic station data quality is commonly defined by metrics such as data completeness or background seismic noise levels in specific frequency bands. However, for temporary networks such as aftershock deployments or induced seismicity monitoring, the most critical metric is often how well the station performs when recording events of interest. A timely measure of station performance can be used for real‐time network maintenance and to help make decisions about which stations may need to be moved or are redundant. We develop new event‐based methods to quickly assess station and network performance, including estimating network magnitude of completeness, determining station signal‐to‐noise ratios as a function of earthquake magnitude, and computing relative station amplitudes. At times, a complete catalog of local seismic events may not exist such as in an aftershock deployment in which hundreds to thousands of small earthquakes may be happening and catalog generation efforts cannot keep up. To overcome this, we use an envelope of the average energy recorded by the network to identify events of interest. We find the log amplitude of events identified using this technique scales linearly with local earthquake magnitudes. We examine two U.S. Geological Survey aftershock networks in Oklahoma to demonstrate this approach can be used to identify poorly performing stations and determine network detection thresholds as early as one day following the deployment of a temporary network.