Fields have uneven surfaces, such as ridges, and the shortwave radiation of fields differs depending on the orientation of the slope of the unevenness, resulting in variations in the distribution of moisture on the ground surface. Therefore, it is necessary to estimate the heat and water balances spatially, taking into account the variation of the moisture distribution on the ground surface. Previously, one-dimensional simulations have been used to estimate the heat and water balance of non-sloping surfaces. To estimate the heat and water balance spatially while taking into account ground surface roughness, it is necessary to first estimate the heat and water balance of the ground surface on the basis of slope orientation. The purpose of this study was to clarify those factors, in addition to shortwave radiation, that affect the heat balance of a bare sloping surface at different orientations. To achieve this, the heat balance calculated using observational data of bare ground, including unevenness, was compared with the heat balance estimated by HYDRUS-1D simulation for each slope orientation. Additionally, the brightness index of RGB images was calculated and compared with the relative ground surface brightness and the estimated heat balance for each slope orientation. The estimated results at night and at sunrise/sunset were extremely small in comparison with the calculated results, and the heat balance simulation in the absence of shortwave radiation remained an issue. The relationship between ground surface brightness and ground conduction heat was completely different depending on slope direction, suggesting that ground surface heat transfer is affected substantially by factors other than shortwave radiation related to slope orientation. The findings indicate that it is necessary to examine the effects of heat transfer in detail to estimate the heat balance related to slope orientation.