Effective Machine Learning - Managing Complexity with MLOps
Machine Learning (ML) and Data Science (DS) have been hot topics for roughly a decade now. A lot has happened in these 10 years: universities have spun up highly specialized Machine Learning study programmes producing skilled engineers and scientists, a remarkable share of organisations has started applying ML in practise and Python has become the Lingua Franca of Data Scientists, offering a highly sophisticated toolstack. Despite this brilliant development over the last decade, organisations keep reporting that ML projects hardly make it out of a proof-of-concept phase and eventually often fall short on delivering business value.
One of the main reasons for this phenomenon are immature ML development processes. Developing Machine Learning software is highly complex. Next to dealing with code, appropriate and reproducible handling of data and models from experimentation to deployment while maximizing iteration speed is paramount. Handling this complexity is difficult and requires a broad set of skills. This is where MLOps (Machine Learning + DevOps) principles and tools come into play.