The hydraulic behavior of soil is determined by its hydraulic properties and their variability in space. In agricultural soils, this heterogeneity may stem from tillage or may have natural origin. The root distribution of plants will adapt to some extent to this soil heterogeneity. However, the combined impact of soil heterogeneity and root water uptake (RWU) on long-term soil water budgets has not received much attention. Numerical experiments helped identify how soil heterogeneity affects plant transpiration, soil evaporation, and groundwater recharge. Two-dimensional virtual soils with hierarchical heterogeneity, both natural and tillage induced, served as a basis for modeling soil water dynamics for a 10-yr climate record from two weather stations in Germany that vastly differ in annual precipitation. The complex interactions between soil and vegetation were explored by (i) comparing different RWU strategies (depth-, structure-, and time-dependent root profiles), (ii) land use types (perennial grass and annual winter crops), (iii) a combination of textures (silt above sand and sand above loam), and (iv) RWU with or without a compensation mechanism. The simulations were repeated with one-dimensional, effective representations of these virtual soils. In the framework of hydropedology, this study shed some light on the interaction between plants and pedological features and its impact on the macroscopic soil water budget. We demonstrated that land use has a major impact on the annual water balance through the partitioning of evapotranspiration into bare soil evaporation and plant transpiration. Compensational RWU becomes important for the annual water balance when the root zone comprises contrasting materials with respect to water holding capacity. Soil heterogeneity has in fact a minor impact on long-term soil water budgets. As a consequence, the relative contribution of plant transpiration, soil evaporation, and groundwater recharge to the total soil water loss was well reproduced by simulations in one-dimensional effective soil profiles. This advocates the application of one-dimensional soil–atmosphere–vegetation transfer (SVAT) models at larger scales. These findings only hold for assumptions made in our numerical simulations including flat area without lateral flow and no macropore flow.