How AI gave us new numbers - A journey on floating point, maths and hardware acceleration

AI models use floating point numbers as internal representation as well as for calculations with vectors,matrices and tensors. Compared to simplistic integer arithmetic this puts a lot of pressure on hardware with regards to available space on the chip, its supported functionality and energy consumption. The talk will give an overview on historically established floating point formats, their preferred usage in machine learning and how new model architectures in parallel with increasing hardware demand gave rise to new floating point formats.