Inference
The phase in machine learning where a trained model is deployed to make predictions or generate outputs on new, previously unseen data in production.
Think of It Like This
Like a student finally taking the final exam after weeks of rigorous studying and practice tests.
During inference, the model's weights are frozen, and computational resources are dedicated entirely to forward passes. Optimizing inference is critical for reducing latency and operational costs in real-world applications. Techniques like quantization, model pruning, and specialized hardware are frequently used to accelerate this process.