Experimental Evaluation of Mixed Precision Training for End-to-End Applications

Deep neural network models are growing rapidly to improve accuracy in many different applications, e.g. language modeling, machine translation, and speech and image recognition. As the models grow, the compute and memory resources to train those models also increase proportionally. Since the growth...