Context Compaction
An optimization strategy that systematically condenses a massive input prompt into a much shorter, dense representation to save both memory and compute time.
Think of It Like This
Like reading an entire textbook and writing down only the most important formulas on a single cheat sheet before walking into a final exam.
Processing a huge document repeatedly in a chatbot session wastes immense computational resources. Techniques like prompt compression use smaller models to extract only the most critical information, while token pruning entirely removes redundant tokens. This drastically reduces the context window footprint without significantly degrading the final answer.