We study the Riemannian Laplace-Beltrami operator $L$ on a Riemannian manifold with Heisenberg group $H_1$ as boundary. We calculate the heat kernel and Green's function for $L$, and give global and small time estimates of the heat kernel. A class of hypersurfaces in this manifold can be regarded as approximations of $H_1$. We also restrict $L$ to each hypersurface and calculate the corresponding heat kernel and Green's function. We will see that the heat kernel and Green's function converge to the heat kernel and Green's function on the boundary.