Olivier talks about software engineering applied to deep learning (regarding the notion of Software 2.0 - https://medium.com/@karpathy/software-2-0-a64152b37c35). He exposes his vision of what a regular software is, and what makes a Deep Learning Software different.
A machine learning instance is composed of:
As a conclusion, he's introducing his Babel Fish: a tool whose goal is to translate mathematical equations (described in Unicode) into computation graph at runtime. For more technical information: go to the Gorgonia repository.