Measurements of the nuclear magnetic resonance (NMR) signal’s behavior with time provide powerful noninvasive insight into the pore-scale environment. The time dependence of the NMR signal, which is a function of parameters called relaxation times, is intimately linked to the geometry of the pore space and has been used successfully to estimate pore size and permeability. The basis for the pore size and permeability estimates is that interactions occurring at the grain surface often function as the primary mechanism controlling the time dependence of the NMR signal. In this limit, called the fast diffusion limit, and when each pore can be considered to be isolated, the measured relaxation times are often interpreted as representative of pore sizes. In heterogeneous media, where the NMR signal is described by a distribution of relaxation times, the measured relaxation time distribution is often interpreted as representative of the underlying pore-size distribution. We have explored a scenario in which an additional relaxation mechanism, which arises due to magnetic field inhomogeneity across the pore space, violates the assumption that interactions occurring at the grain surface are the dominant relaxation mechanism. Using both synthetic and laboratory studies, we demonstrate that magnetic field inhomogeneity can lead to a complex relationship between the measured relaxation time distribution and the underlying pore-size distribution. Magnetic field inhomogeneity is observed to lead to a spatially heterogeneous magnetization density across the pore space requiring multiple eigenmodes to describe the evolution of the magnetization within a single pore during the NMR experiment. This results in a breakdown of the validity of the interpretation of the relaxation time distribution as representative of the underlying pore-size distribution for sediments with high magnetic susceptibility.