The variogram is a key parameter for geostatistical modelling. Inferring a stable variogram model from widely spaced well data is a longstanding challenge due to an often unreliable experimental horizontal variogram. The main aim of this paper is to improve the horizontal variogram inference in the presence of limited data by quantifying variogram uncertainty and reducing this uncertainty with secondary data. A new approach of variogram uncertainty is presented by computing the number of independent variogram pairs (degrees of freedom) for each lag. A methodology to improve the horizontal variogram uncertainty is developed considering the horizontal variogram of the seismic data and the vertical well variogram since these variograms are well defined in most cases. Seismic data provide constraints on the horizontal variogram of the well data. The constraints are inferred from the covariance between the well and seismic data. The vertical variogram of the well data can be scaled to scenarios of the horizontal variogram. Improved horizontal variogram realizations honouring the correlation between lags are attained by merging variogram distributions for each lag distance considering the constraints from the horizontal seismic variogram. A realistic case study is presented.