Many electrical and electromagnetic (EM) methods operate at MHz frequencies, at which the interfacial polarization occurring at the solid-liquid interface in geologic materials may dominate the electrical signals. To correctly interpret electrical/EM measurements, it is therefore critical to understand how the interfacial polarization influences the effective electrical conductivity and permittivity spectra of geologic materials. We have used pore-scale simulation to study the role of material texture and packing in interfacial polarization in water-saturated granular soils. Synthetic samples with varying material textures and packing densities are prepared with the discrete element method. The effective electrical conductivity and permittivity spectra of these samples are determined by numerically solving the Laplace equation in a representative elementary volume of the samples. The numerical results indicate that the effective permittivity of granular soils increases as the frequency decreases due to the polarizability enhancement from the interfacial polarization. The induced permittivity increment is mainly influenced by the packing state of the samples, increasing with the packing density. Material textures such as the grain shape and size distribution may also affect the permittivity increment, but their effects are less significant. The frequency characterizing the interfacial polarization (i.e., the characteristic frequency) is mainly related to the electrical contrast of the solid and water phases. The model based on the traditional differential effective medium (DEM) theory significantly underestimates the permittivity increment by a factor of more than two and overestimates the characteristic frequency by approximately 1 MHz. These inaccurate predictions are due to the fact that the electrical interactions between neighboring grains are not considered in the DEM theory. A simple empirical equation is suggested to scale up the theoretical depolarization factor of grains entering the DEM theory to account for the interaction of neighboring grains in granular soils.