We present a Bayesian approach for simultaneous estimation of high-dimensional conductivity fields and associated variograms. Our approach merges periodic embedding with dimensionality reduction to decouple the variogram from the random numbers, and facilitate MCMC simulation. Using the Matérn variogram allows for inference of the field smoothness. Conditioning on direct measurements is straightforward. We illustrate our method using a synthetic flow and transport experiment involving a 10,000-dimensional conductivity field. A 40-times reduction of the size of the parameter space did not prevent the posterior simulations to appropriately fit the measurement data and the posterior parameter distributions to include the true geostatistical parameter values. Overall, the posterior field realizations covered a wide range of geostatistical models, questioning the common practice of assuming a fixed variogram prior to inference of the hydraulic conductivity values. Our method is shown to be more efficient than sequential Gibbs sampling (SGS) for the considered case study, particularly when implemented on a distributed computing cluster. It is also found to outperform the method of anchored distributions (MAD) for the same computational budget.