If $\mathfrak{A}$ is a finite alphabet, $\sU \subset\mathbb{Z}^D$, and $\mu_\sU$ is a probability measure on $\mathfrak{A}^\sU$ that ``looks like'' the marginal projection of a stationary stochastic process on $\mathfrak{A}^{\mathbb{Z}^D}$, then can we ``extend'' $\mu_\sU$ to such a process? Under what conditions can we make this extension ergodic, (quasi)periodic, or (weakly) mixing? After surveying classical work on this problem when $D=1$, we provide some sufficient conditions and some necessary conditions for $\mu_\sU$ to be extendible for $D>1$, and show that, in general, the problem is not formally decidable.