Epidemic‐type aftershock sequence (ETAS) catalogs generated from the third Uniform California Earthquake Rupture Forecast (UCERF3) model are unique in that they are the first to combine a complex, fault‐based long‐term forecast with short‐term earthquake clustering statistics. We present Turing‐style tests to examine whether these synthetic catalogs can successfully imitate observed earthquake behavior in California. We find that UCERF3‐ETAS is more spatially diffuse than the observed historic catalog in California and that it lacks quiet periods that are present in the real catalog. Although mean aftershock productivity of the observed catalog is matched closely by UCERF3‐ETAS, the real catalog has more intersequence productivity variability and small mainshocks have more foreshocks. In sum, we find that UCERF3‐ETAS differs from the observed catalog in ways that are foreseeable from its modeling simplifications. The tests we present here can be used on any model that produces suites of synthetic catalogs; as such, in addition to providing avenues for future improvements to the model, they could be incorporated into testing platforms such as Collaboratory for the Study of Earthquake Predictability (CSEP).