CircuitFlow: : A Domain Specific Language for Dataflow ProgrammingIn-Person
Dataflow applications, such as machine learning algorithms, can run for days, making it desirable to have assurances that they will work correctly. Current tools are not good enough: too often the interactions between tasks are not type-safe, leading to undesirable runtime errors.This paper presents a new declarative Haskell embedded DSL (eDSL) for dataflow programming: CircuitFlow. Defined as a Symmetric Monoidal Preorder on data that models dependencies in the workflow, it has a strong mathematical basis, refocusing on how data flows through an application, resulting in a more expressive solution that not only catches errors statically, but also achieves competitive run-time performance. In our preliminary evaluation, CircuitFlow outperforms the industry-leading Luigi library of Spotify by scaling better with the number of inputs.
Tue 18 JanDisplayed time zone: Eastern Time (US & Canada) change
13:30 - 14:45 | |||
13:30 25mTalk | Graph-based Interpretation of Normal Logic ProgramsRemote PADL Fang Li The University of Texas at Dallas, Elmer Salazar The University of Texas at Dallas, Gopal Gupta The University of Texas at Dallas | ||
13:55 25mTalk | Functional Programming on Top of SQL EnginesRemote PADL Tobias Burghardt University of Tübingen, Denis Hirn University of Tübingen, Torsten Grust University of Tübingen | ||
14:20 25mTalk | CircuitFlow: : A Domain Specific Language for Dataflow ProgrammingIn-Person PADL Riley Evans University of Bristol, Samantha Frohlich University of Bristol, Meng Wang University of Bristol |