CUDA
A parallel computing platform and programming interface created by NVIDIA that allows software developers to directly access the immense power of modern GPUs.
Think of It Like This
Like finally getting the master key to a factory full of thousands of workers, allowing you to give them all instructions simultaneously instead of one by one.
Before CUDA, GPUs were strictly used for rendering graphics for video games. By exposing a C++ interface to the thousands of tiny, highly parallel cores on a graphics card, CUDA inadvertently sparked the deep learning revolution. Today, almost all major machine learning frameworks like PyTorch and TensorFlow rely heavily on CUDA to accelerate matrix multiplications.