Many observations suggest that relatively small changes in stress may trigger or inhibit earthquakes. This result is surprising because the perturbing stress is only a small fraction of the stress drop from the affected earthquake. We investigate this phenomenon by simulating the earthquake fault with a numerical model that is physically realistic. A fault is not homogeneous, and therefore, the failure criteria will vary over the surface of the fault. We model fault heterogeneity as simple Coulomb failure on smaller but mostly uniform areas that comprise the fault. As a result of multiple (and repeated) failures over these small areas, the fault can have a very uneven stored stress. We show that it is possible for small changes in confining stress of a few tenths of a bar to advance or retard the occurrence of large earthquakes by decades. Because such stress changes can be generated by nearby earthquakes, and over much longer time scales by postseismic strain diffusion from large distant earthquakes, estimates of the probabilities of earthquake occurrence must take into account these stress perturbations in order to be reliable.