Snapchat Shows How It Uses GPUs To Accelerate Machine Learning (ML) Inferences

Source: https://eng.snap.com/applying_gpu_to_snap/ Machine learning (ML) and Artificial intelligence (AI) have transformed how industries make business decisions. Many firms are now leveraging ML and AI to compile consumer data and analyze and predict future consumer behaviour. This has allowed them to process high volumes of data rapidly and accurately and analyze valuable insights to take promising actions for their business.  In their recent blogs, Snap shares their experience of applying GPU technology to accelerate ML model inference. The inference is the computation-intensive process of calculating model predictions (like the probability of a Snapchatter watching the complete video) from input features (like the number of videos viewed by Snapchatter in the past hour).  This is a challenging task for firms like Snap, having a community of over 293 million Snapchatters daily and creating over 10 trillion ML predictions daily.  ML models on Snapchat are based on deep neural networks (DNN), which makes them so accurate but also computationally heavy. They use a highly scalable and efficient inference stack within the x86-64 CPU ecosystem to overcome this problem. And soon after the launch of inference-oriented accelerators like the NVidia T4 GPU device, they started to investigate whether it can offer a better tradeoff…
Read More