Probabilistic and differentiable programming paradigms are being adopted by the scientific community, promising major advances in simulation pipelines, data analysis, and design optimization of experiments. This talk will cover ongoing work in the probabilistic programming and differentiable programming domains with a focus on large-scale scientific applications. Probabilistic programming languages (PPLs) allow us to specify complex generative models as computer code and perform Bayesian inference in these models automatically. However, applications of these languages in science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the lack of scalable implementations. To address these, we present a novel probabilistic programming framework that connects to existing simulators through a cross-platform protocol, which we call the Probabilistic Programming eXecution (PPX) protocol, and provides random-walk Metropolis–Hastings and inference compilation engines for tractable inference. We show how PPX allows any PPL system to be coupled to any given simulator so that these two sides can be (1) implemented in different programming languages and (2) executed in separate processes and possibly on separate machines across a network connection. We show examples of the approach in particle physics, spacecraft collision avoidance, and simulation of composite materials. We discuss extensions such as replacing the simulator with a fast deep-learning-based surrogate making use of PPX and retaining the original address structure and stochastic control flow of the simulator, and adding differentiation support to PPX. We also talk about ongoing efforts to use differentiable programming for design optimization of future experiments at CERN. The talk with also cover news about DiffSharp, a differentiable programming library in F#.