Probabilistic programming strives to make statistical analysis more accessible by separating probabilistic modelling from probabilistic inference. In practice this decoupling is difficult. Different inference techniques are applicable to different classes of models, have different advantages and shortcomings, and require different optimisation and diagnostics techniques to ensure robustness and reliability. No single inference algorithm can be used as a probabilistic programming back-end that is simultaneously reliable, efficient, black-box, and general. Probabilistic programming languages often choose a single algorithm to apply to a given problem, thus inheriting its limitations. This talk advocates for using program analysis to make better use of the available structure in probabilistic programs, and thus better utilising the underlying inference algorithm. I will show several techniques, which analyse a probabilistic program and adapt it to make inference more efficient, sometimes in a way that would have been tedious or impossible to do by hand.