We revisit with another view point the construction by R. Brylinski and B. Kostant of minimal representations of simple Lie groups. We start from a pair $(V,Q)$, where $V$ is a complex vector space and $Q$ a homogeneous polynomial of degree 4 on $V$. The manifold $\Xi $ is an orbit of a covering of ${\rm Conf}(V,Q)$, the conformal group of the pair $(V,Q)$, in a finite dimensional representation space. By a generalized Kantor-Koecher-Tits construction we obtain a complex simple Lie algebra $\mathfrak g$, and furthermore a real form ${\mathfrak g}_{\mathbb R}$. The connected and simply connected Lie group $G_{\mathbb R}$ with ${\rm Lie}(G_{\mathbb R})={\mathfrak g}_{\mathbb R}$ acts unitarily on a Hilbert space of holomorphic functions defined on the manifold $\Xi $.