Serverless Practitioners Summit Europe 2020

Putting machine learning models into production is a challenging task for data scientists and devops within an organization. In this talk we will discuss how KFServing builds on top of the KNative project to provide serverless machine learning inference. It provides simple abstractions for deploying standard and custom machine learning models and updating them via canaries. Serverless technology helps to provide painless auto-scaling and scale to zero functionality over complex GPU and CPU based components. However, serverless and machine learning inference do have their challenges especially in latency critical environments and we will discuss some of those challenges as well as the future prospects for the use of serverless within machine learning.