A recurring problem in heterotic compactifications is the plethora of moduli fields in the resulting low-energy theories which we do not observe in nature. One might be able to lift these moduli by moving to non-Kahler compactifications. The general $N=1$ heterotic solution with a 4d Minkowski vacuum is described by the Strominger system. The compactification manifold is non-Kahler, there is H flux and one has a non-trivial Bianchi identity to deal with. The moduli of these solutions has been a mystery until recently. I will present work on understanding the moduli of these compactifications to higher orders using the heterotic superpotential. Obstructions to integrating the deformations appear as non-zero Yukawa couplings in the low-energy theory. I will also comment on links to generalised geometry and a generalisation of Kodaira-Spencer gravity.