We study the optimisation of expectations, a problem which arises for examples in variational inference using the reparametrisation trick. As our starting point we take a variant of the simply-typed lambda calculus with reals and conditionals to express the function the expectation of which is taken. Unfortunately, it is well-known that the reparametrisation gradient estimator is biased in this setting due to fact that the model is not differentiable in general. As a consequence, stochastic gradient descent, which is commonly used to solve such problems, may converge to incorrect results. We have devised a type system restricting the language, and designed a stochastic gradient descent-like procedure for solving the optimisation problem which provably converges to stationary points for typable programs. Our approach is based on a smoothed denotational semantics in the cartesian closed category of Frölicher spaces.